Commit Graph

5832 Commits

Author SHA1 Message Date
Pekka Enberg
233beeb8e7 Merge 'core: Wrap Connection::schema in RwLock' from Pekka Enberg
Closes #3261
2025-09-23 11:50:24 +03:00
Pekka Enberg
b94aa22499 core: Wrap Connection::schema in RwLock 2025-09-23 10:31:20 +03:00
Pavan-Nambi
59660f0c25 remove tests for column type spaces 2025-09-23 12:48:35 +05:30
Pavan-Nambi
8e02855c98 update seq table onconflict nothing too and refactor logic into seperate function
add testcase for failed fuzz

more

remove autoincrement if col aint integer

fmt
2025-09-23 11:41:52 +05:30
Pekka Enberg
9d395a5a52 Merge 'Stop incrementing n_changes for idx delete' from Preston Thorpe
closes #3259
Previously we were emitting the following:
<img width="483" height="135" alt="image" src="https://github.com/user-
attachments/assets/e12100ed-5815-4619-829a-3230eb8c8f7f" />
After:
<img width="484" height="310" alt="image" src="https://github.com/user-
attachments/assets/28591f52-18b1-4060-8c92-7a3f7194fca0" />

Closes #3258
2025-09-23 07:30:41 +03:00
Pekka Enberg
b857f94fe4 Merge 'core: Wrap Connection::pager in RwLock' from Pekka Enberg
Closes #3247
2025-09-23 07:29:09 +03:00
PThorpe92
9238584a75 Stop incrementing n_changes for idx delete 2025-09-22 19:10:35 -04:00
Pavan-Nambi
0854bd3e9a clippy error 2025-09-22 22:39:59 +05:30
Pavan Nambi
837d441861 Update core/translate/insert.rs
Co-authored-by: Preston Thorpe <preston@unlockedlabs.org>
2025-09-22 22:35:31 +05:30
Pavan-Nambi
57805e0f6a fmt 2025-09-22 21:13:26 +05:30
Pavan Nambi
f1ac855441 Merge branch 'main' into cdc_fail_autoincrement 2025-09-22 21:11:26 +05:30
Pavan-Nambi
215307a9bd cleanup - remove comments 2025-09-22 21:08:02 +05:30
PThorpe92
10662ee5c5 Fix error in test missing DatabaseOpts field 2025-09-22 11:28:20 -04:00
PThorpe92
8420b9be04 Disable runtime extension loading unless enabled 2025-09-22 11:28:19 -04:00
PThorpe92
4ac4aff30c Add a flag to DatabaseOpts, only for cli_only feature to enable rt extension loading 2025-09-22 11:28:19 -04:00
PThorpe92
d2c643da06 Add cli_only feature to core 2025-09-22 11:28:19 -04:00
Jussi Saurio
1d4b301f05 Merge 'mvcc: simplify StateMachine' from Jussi Saurio
`TransitionResult::Continue` is an internal implementation detail that
tells an invocation of `StateMachine::step()` to continue looping, but
it is of no use to upstream callers.
For this reason, just return an IOResult from StateMachine::step() which
simplifies the result handling.

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #3248
2025-09-22 17:03:16 +03:00
Pekka Enberg
aa454a6637 core: Wrap Connection::pager in RwLock 2025-09-22 17:02:08 +03:00
Pekka Enberg
69b2e86c9c Merge 'Fix busy handler' from Lâm Hoàng Phúc
@penberg i think it fixed #3144, but i got locked database error
```sh
Running write throughput benchmark with 5 threads, 1000 batch size, 1000 iterations, mode: Legacy
Database created at: write_throughput_test.db
Thread error 0: SQL execution failure: `database is locked`
Thread 1: 1000000 inserts in 514.45s (1943.82 inserts/sec)
Error: SqlExecutionFailure("database is locked")
```

Closes #3147
2025-09-22 16:43:51 +03:00
Jussi Saurio
4af49ef98c mvcc: simplify StateMachine
TransitionResult is an internal implementation detail that tells
an invocation of StateMachine::step() to continue looping, but it
is of no use to other callers.

For this reason, just return an IOResult from StateMachine::step()
which simplifies the result handling.
2025-09-22 16:37:31 +03:00
Jussi Saurio
5498816d0b Merge 'mvcc: add blocking checkpoint lock' from Jussi Saurio
This PR does not implement MVCC checkpoint yet, just adds a lock for it.
MVCC checkpoints are always TRUNCATE, plus they block all other
transactions. This guarantees that never need to let transactions read
from the SQLite WAL.
In MVCC, the checkpoint procedure is roughly as follows:
- Take the blocking_checkpoint_lock
- Write everything in the logical log to the pager, and from there
commit to the SQLite WAL.
- Immediately TRUNCATE checkpoint the WAL into the database file.
- Release the blocking_checkpoint_lock.

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #3244
2025-09-22 16:16:26 +03:00
Preston Thorpe
44dc4c9636 Merge 'translate/emitter: Implement partial indexes' from Preston Thorpe
This PR adds support for partial indexes, e.g. `CREATE INDEX` with a
provided predicate
```sql
CREATE UNIQUE INDEX idx_expensive ON products(sku) where price > 100;
```
The PR does not yet implement support for using the partial indexes in
the optimizer.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3228
2025-09-22 09:09:54 -04:00
Pavan-Nambi
8dc485e5f7 don't allow duplicate columns and get column type more precisely 2025-09-22 17:39:08 +05:30
Pekka Enberg
372daef656 core: Wrap Pager::io_ctx in RwLock 2025-09-22 15:00:29 +03:00
Pekka Enberg
2af98223ae Merge 'Enable checksum tests if checksum feature is on' from Kacper Kołodziej
These tests fail if checksum feature is turned off.

Closes #3242
2025-09-22 14:40:34 +03:00
Pere Diaz Bou
2f4426fc33 core/mvcc/logical-log: change order of Data size in InsertRow 2025-09-22 13:25:46 +02:00
Pere Diaz Bou
4cc88ee2bb core/mvcc/logical-log: rename Insert and Delete -> InsertRow and DeleteRow in LogRecordType 2025-09-22 13:21:54 +02:00
Pere Diaz Bou
db326affc6 core/mvcc/logical-log: rename LogRowType to LogRecordType 2025-09-22 13:19:02 +02:00
Pere Diaz Bou
e22a3893d5 core/mvcc/logical-log: remove column_count from insert row type 2025-09-22 13:18:17 +02:00
Pere Diaz Bou
4c959e760b core/mvcc/logical-log: add rows size field for transaction format 2025-09-22 13:15:07 +02:00
Pere Diaz Bou
2cd1562966 core/mvcc/logical-log: add format for transaction fields and marker end 2025-09-22 13:06:11 +02:00
Pere Diaz Bou
6fc1bed187 core/mvcc/logical-log: add format for logical log header 2025-09-22 13:00:52 +02:00
Pekka Enberg
49f551e6c3 Merge 'Wrap Pager vacuum state in RwLock' from Pekka Enberg
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3239
2025-09-22 13:44:22 +03:00
Pere Diaz Bou
36a728d984 core/mvcc/logical-log: add format for row types 2025-09-22 12:43:54 +02:00
Jussi Saurio
6a20735fe0 mvcc: add blocking checkpoint lock
MVCC checkpoints are always TRUNCATE, plus they block all other transactions.
This guarantees that never need to let transactions read from the SQLite WAL.

In MVCC, the checkpoint procedure is roughly as follows:
- Take the blocking_checkpoint_lock
- Write everything in the logical log to the pager, and from there commit to the SQLite WAL.
- Immediately TRUNCATE checkpoint the WAL into the database file.
- Release the blocking_checkpoint_lock.
2025-09-22 12:40:19 +03:00
TcMits
17c91c1fe2 resolve conflicts 2025-09-22 16:03:52 +07:00
Kacper Kołodziej
76f2e4e217 Enable checksum tests if checksum feature is on
These tests fail if checksum feature is turned off.
2025-09-22 10:46:49 +02:00
Pekka Enberg
37866e74e5 Merge 'core/io: Ensure callbacks are invoked once' from Pedro Muniz
Add a `Once` object to uphold this property. We cannot use the OnceLock
to dictate this, because if we set the OnceLock before actually calling
the callback, there is a moment in time where we will have an incorrect
transient state. This change ensures we atomically call the callback and
then set the OnceLock
Should fix #3217
Closes #3217

Closes #3237
2025-09-22 11:44:39 +03:00
Pekka Enberg
0144ea8059 Merge 'Support UNION queries in DBSP-based Materialized Views' from Glauber Costa
UNION queries, while useful on their own, are a cornerstone of recursive
CTEs.
This PR implements:
* the merge operator, required to merge both sides of a union query.
* the circuitry necessary to issue the Merge operator.
* extraction of tables mentioned in union and CTE expressions, so we can
correctly populate tables that contain them.

Closes #3234
2025-09-22 11:33:19 +03:00
Pekka Enberg
19849c7895 Merge 'DBSP: Return a parse error for a non-equality join' from Glauber Costa
We currently don't handle non equality, but end up just returning a
bogus result. Let's parse error.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3232
2025-09-22 11:32:43 +03:00
Mayank Verma
7505a2a567 fix CTE alias resolution in planner 2025-09-22 13:51:48 +05:30
Pekka Enberg
6f258b37d9 core/storage: Wrap Pager vacuum state in RwLock 2025-09-22 10:34:58 +03:00
Jussi Saurio
c0fc2ad234 fix optimizer tests 2025-09-22 10:18:03 +03:00
Jussi Saurio
eada24b508 Store in-memory index definitions most-recently-seen-first
This solves an issue where an INSERT statement conflicts with
multiple indices. In that case, sqlite iterates the linked list
`pTab->pIndex` in order and handles the first conflict encountered.
The newest parsed index is always added to the head of the list.

To be compatible with this behavior, we also need to put the most
recently parsed index definition first in our indexes list for a given
table.
2025-09-22 10:11:50 +03:00
Pekka Enberg
f053b76518 core/storage: Move vacuum state machines to VacuumState 2025-09-22 09:37:40 +03:00
Pekka Enberg
6280cfc59d Merge branch 'main' into sync-improvements 2025-09-22 07:35:39 +03:00
Pekka Enberg
dae72a7418 Merge 'Remove some unnecessary unsafe impls' from Pedro Muniz
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3236
2025-09-22 07:33:50 +03:00
Glauber Costa
2627ad44de support union statements in the DBSP circuit compiler 2025-09-21 21:00:27 -03:00
Glauber Costa
b419db489a Implement the DBSP merge operator
The Merge operator is a stateless operator that merges two deltas.
There are two modes: Distinct, where we merge together values that
are the same, and All, where we preserve all values. We use the rowid of
the hashable row to guarantee that: In Distinct mode, the rowid is set
to 0 in both sides. If they values are the same, they will hash to the
same thing. For All, the rowids are different.

The merge operator is used for the UNION statement, which is a
cornerstone of Recursive CTEs.
2025-09-21 21:00:27 -03:00
Glauber Costa
9f54f60d45 make sure that complex select statements are captured by MV populate
The population code extracts table information from the select statement
so it can populate the materialized view. But the code, as written
today, is naive. It doesn't capture table information correctly if there
is more than one select statement (such in the case of a union query).
2025-09-21 21:00:27 -03:00