Commit Graph

10455 Commits

Author SHA1 Message Date
Pavan-Nambi
57805e0f6a fmt 2025-09-22 21:13:26 +05:30
Pavan Nambi
f1ac855441 Merge branch 'main' into cdc_fail_autoincrement 2025-09-22 21:11:26 +05:30
Pavan-Nambi
215307a9bd cleanup - remove comments 2025-09-22 21:08:02 +05:30
PThorpe92
10662ee5c5 Fix error in test missing DatabaseOpts field 2025-09-22 11:28:20 -04:00
PThorpe92
0c54c2b255 Add turos_cli option to CLI DatabaseOpts 2025-09-22 11:28:19 -04:00
PThorpe92
8420b9be04 Disable runtime extension loading unless enabled 2025-09-22 11:28:19 -04:00
PThorpe92
4ac4aff30c Add a flag to DatabaseOpts, only for cli_only feature to enable rt extension loading 2025-09-22 11:28:19 -04:00
PThorpe92
cfa89c9ddb use cli_only feature in CLI Cargo.toml 2025-09-22 11:28:19 -04:00
PThorpe92
d2c643da06 Add cli_only feature to core 2025-09-22 11:28:19 -04:00
Jussi Saurio
1d4b301f05 Merge 'mvcc: simplify StateMachine' from Jussi Saurio
`TransitionResult::Continue` is an internal implementation detail that
tells an invocation of `StateMachine::step()` to continue looping, but
it is of no use to upstream callers.
For this reason, just return an IOResult from StateMachine::step() which
simplifies the result handling.

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #3248
2025-09-22 17:03:16 +03:00
Pekka Enberg
aa454a6637 core: Wrap Connection::pager in RwLock 2025-09-22 17:02:08 +03:00
Pavan-Nambi
cd1f1ce46e add tests for duplicate columns and column type
fmt

correct expected values spaces
2025-09-22 19:19:06 +05:30
Pekka Enberg
69b2e86c9c Merge 'Fix busy handler' from Lâm Hoàng Phúc
@penberg i think it fixed #3144, but i got locked database error
```sh
Running write throughput benchmark with 5 threads, 1000 batch size, 1000 iterations, mode: Legacy
Database created at: write_throughput_test.db
Thread error 0: SQL execution failure: `database is locked`
Thread 1: 1000000 inserts in 514.45s (1943.82 inserts/sec)
Error: SqlExecutionFailure("database is locked")
```

Closes #3147
2025-09-22 16:43:51 +03:00
Jussi Saurio
4af49ef98c mvcc: simplify StateMachine
TransitionResult is an internal implementation detail that tells
an invocation of StateMachine::step() to continue looping, but it
is of no use to other callers.

For this reason, just return an IOResult from StateMachine::step()
which simplifies the result handling.
2025-09-22 16:37:31 +03:00
Jussi Saurio
5498816d0b Merge 'mvcc: add blocking checkpoint lock' from Jussi Saurio
This PR does not implement MVCC checkpoint yet, just adds a lock for it.
MVCC checkpoints are always TRUNCATE, plus they block all other
transactions. This guarantees that never need to let transactions read
from the SQLite WAL.
In MVCC, the checkpoint procedure is roughly as follows:
- Take the blocking_checkpoint_lock
- Write everything in the logical log to the pager, and from there
commit to the SQLite WAL.
- Immediately TRUNCATE checkpoint the WAL into the database file.
- Release the blocking_checkpoint_lock.

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #3244
2025-09-22 16:16:26 +03:00
Preston Thorpe
44dc4c9636 Merge 'translate/emitter: Implement partial indexes' from Preston Thorpe
This PR adds support for partial indexes, e.g. `CREATE INDEX` with a
provided predicate
```sql
CREATE UNIQUE INDEX idx_expensive ON products(sku) where price > 100;
```
The PR does not yet implement support for using the partial indexes in
the optimizer.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3228
2025-09-22 09:09:54 -04:00
Pekka Enberg
972c01f6f7 Merge 'core: Wrap Pager::io_ctx in RwLock' from Pekka Enberg
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3246
2025-09-22 15:41:30 +03:00
Pavan-Nambi
8dc485e5f7 don't allow duplicate columns and get column type more precisely 2025-09-22 17:39:08 +05:30
Pekka Enberg
372daef656 core: Wrap Pager::io_ctx in RwLock 2025-09-22 15:00:29 +03:00
Pekka Enberg
2af98223ae Merge 'Enable checksum tests if checksum feature is on' from Kacper Kołodziej
These tests fail if checksum feature is turned off.

Closes #3242
2025-09-22 14:40:34 +03:00
Pere Diaz Bou
2f4426fc33 core/mvcc/logical-log: change order of Data size in InsertRow 2025-09-22 13:25:46 +02:00
Pere Diaz Bou
4cc88ee2bb core/mvcc/logical-log: rename Insert and Delete -> InsertRow and DeleteRow in LogRecordType 2025-09-22 13:21:54 +02:00
Pere Diaz Bou
db326affc6 core/mvcc/logical-log: rename LogRowType to LogRecordType 2025-09-22 13:19:02 +02:00
Pere Diaz Bou
e22a3893d5 core/mvcc/logical-log: remove column_count from insert row type 2025-09-22 13:18:17 +02:00
Pere Diaz Bou
4c959e760b core/mvcc/logical-log: add rows size field for transaction format 2025-09-22 13:15:07 +02:00
Pere Diaz Bou
2cd1562966 core/mvcc/logical-log: add format for transaction fields and marker end 2025-09-22 13:06:11 +02:00
Pere Diaz Bou
6fc1bed187 core/mvcc/logical-log: add format for logical log header 2025-09-22 13:00:52 +02:00
Pekka Enberg
49f551e6c3 Merge 'Wrap Pager vacuum state in RwLock' from Pekka Enberg
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3239
2025-09-22 13:44:22 +03:00
Pere Diaz Bou
36a728d984 core/mvcc/logical-log: add format for row types 2025-09-22 12:43:54 +02:00
Jussi Saurio
127bc471ff Merge 'Enhancement to Sim Snapshot isolation code' from Pedro Muniz
- Made the code around snapshot isolation more ergonomic with each
connections having its own transaction state. Also when shadowing, we
pass a ShadowTablesMut object that dynamically uses the committed tables
or the connection tables depending on the transaction state.
- added begin concurrent transaction before every property when mvcc is
enabled (this is just so we can have some mvcc code be tested using the
simulator under Begin Concurrent, I have not yet implemented the logic
to have concurrent transactions in the simulator)
- some small enhancements to shrinking
TODOs
- have proper logic to have concurrent transactions without WriteWrite
conflicts. This means when generating the plans we need to make sure
that we do not generate rows that will conflict with rows in other
transactions. This is slightly more powerful than what we do in the
fuzzer, as we just have `WriteWriteConflict` as an acceptable error
there. By baking this `NoConflict` approach to the simulator, we can
continuously test the what does not trigger a WriteWriteConflict and
snapshot isolation.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3226
2025-09-22 12:48:36 +03:00
Jussi Saurio
6a20735fe0 mvcc: add blocking checkpoint lock
MVCC checkpoints are always TRUNCATE, plus they block all other transactions.
This guarantees that never need to let transactions read from the SQLite WAL.

In MVCC, the checkpoint procedure is roughly as follows:
- Take the blocking_checkpoint_lock
- Write everything in the logical log to the pager, and from there commit to the SQLite WAL.
- Immediately TRUNCATE checkpoint the WAL into the database file.
- Release the blocking_checkpoint_lock.
2025-09-22 12:40:19 +03:00
TcMits
17c91c1fe2 resolve conflicts 2025-09-22 16:03:52 +07:00
Kacper Kołodziej
76f2e4e217 Enable checksum tests if checksum feature is on
These tests fail if checksum feature is turned off.
2025-09-22 10:46:49 +02:00
Pekka Enberg
37866e74e5 Merge 'core/io: Ensure callbacks are invoked once' from Pedro Muniz
Add a `Once` object to uphold this property. We cannot use the OnceLock
to dictate this, because if we set the OnceLock before actually calling
the callback, there is a moment in time where we will have an incorrect
transient state. This change ensures we atomically call the callback and
then set the OnceLock
Should fix #3217
Closes #3217

Closes #3237
2025-09-22 11:44:39 +03:00
Pekka Enberg
26f90257a6 Turso 0.2.0-pre.6 2025-09-22 11:44:21 +03:00
Pekka Enberg
979e2542af Merge branch 'main' of https://github.com/tursodatabase/limbo 2025-09-22 11:44:07 +03:00
Mayank Verma
14da972358 add test case for CTE with alias 2025-09-22 14:13:14 +05:30
Pekka Enberg
f4b0fb17f7 Turso 0.2.0-pre.5 2025-09-22 11:34:22 +03:00
Pekka Enberg
0144ea8059 Merge 'Support UNION queries in DBSP-based Materialized Views' from Glauber Costa
UNION queries, while useful on their own, are a cornerstone of recursive
CTEs.
This PR implements:
* the merge operator, required to merge both sides of a union query.
* the circuitry necessary to issue the Merge operator.
* extraction of tables mentioned in union and CTE expressions, so we can
correctly populate tables that contain them.

Closes #3234
2025-09-22 11:33:19 +03:00
Pekka Enberg
19849c7895 Merge 'DBSP: Return a parse error for a non-equality join' from Glauber Costa
We currently don't handle non equality, but end up just returning a
bogus result. Let's parse error.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3232
2025-09-22 11:32:43 +03:00
Pekka Enberg
390181c024 Merge 'Upgrade dist to 0.30.0' from Pekka Enberg
Closes #3241
2025-09-22 11:30:51 +03:00
Mayank Verma
7505a2a567 fix CTE alias resolution in planner 2025-09-22 13:51:48 +05:30
Pekka Enberg
6f258b37d9 core/storage: Wrap Pager vacuum state in RwLock 2025-09-22 10:34:58 +03:00
Jussi Saurio
f1b0ffc053 clippy is angry about printing an empty string 2025-09-22 10:21:16 +03:00
Pekka Enberg
e5a3512f79 Upgrade dist to 0.30.0 2025-09-22 10:19:46 +03:00
Jussi Saurio
c0fc2ad234 fix optimizer tests 2025-09-22 10:18:03 +03:00
Jussi Saurio
eada24b508 Store in-memory index definitions most-recently-seen-first
This solves an issue where an INSERT statement conflicts with
multiple indices. In that case, sqlite iterates the linked list
`pTab->pIndex` in order and handles the first conflict encountered.
The newest parsed index is always added to the head of the list.

To be compatible with this behavior, we also need to put the most
recently parsed index definition first in our indexes list for a given
table.
2025-09-22 10:11:50 +03:00
Jussi Saurio
ef9f2f9a33 test/fuzz: add prints to get exact executed statements for debugging 2025-09-22 10:11:42 +03:00
Pekka Enberg
751261566e Fix Antithesis and simulator Dockerfiles 2025-09-22 10:02:01 +03:00
Pekka Enberg
f053b76518 core/storage: Move vacuum state machines to VacuumState 2025-09-22 09:37:40 +03:00