Commit Graph

5767 Commits

Author SHA1 Message Date
pedrocarlo
b00a276960 add scoped locking for SharedWalFile to avoid holding locks for longer than needed 2025-10-20 10:45:14 -03:00
Pekka Enberg
97991a1934 Merge 'Fix deferred FK violations check before committing to WAL' from Jussi Saurio
DEFERRED was a bit too deferred - it allowed the dirty pages to be
written out to WAL before checking for violations, resulting in the
violations effectively being committed even though the transaction ended
up aborting
Closes #3784

Closes #3785
2025-10-20 14:51:25 +03:00
Jussi Saurio
10532544dc Fix: check deferred FK violations before committing to WAL
DEFERRED was a bit too deferred - it allowed the dirty pages to be
written out to WAL before checking for violations, resulting in the
violations effectively being committed even though the transaction
ended up aborting
2025-10-20 14:00:49 +03:00
Pekka Enberg
e03f6dbf94 core/storage: Reduce logging level 2025-10-17 20:09:00 +03:00
Jussi Saurio
56e9ba9eae Merge 'WAL: don't hold shared lock across IO operations' from Jussi Saurio
Without this change and running:
```
cd stress
while cargo run -- --nr-threads=4 -i 1000 --verbose --busy-timeout=0; do; done
```
I can produce a deadlock quite reliably.
With this change, I can't.
Even with 5 second busy timeout (the default), the run makes progress
although it is slow as hell because of the busy timeout.
Full disclosure: i couldn't figure out based on parking lot RwLock
semantics why this would fix it, so maybe it just lessens the
probability

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3759
2025-10-17 08:50:26 +03:00
PThorpe92
ddd674c340 Move duplicate table identifier checking to parse_join to allow for natural joins 2025-10-16 18:32:48 -04:00
PThorpe92
10c69b910e Prevent ambiguous self-join table reference 2025-10-16 16:39:10 -04:00
PThorpe92
edaa1b675e Prevent column definitions on CREATE TABLE or opening DB with ON CONFLICT on column def 2025-10-16 15:45:20 -04:00
Jussi Saurio
2ca388d78d WAL: don't hold shared lock across IO operations
Without this change and running:

```
cd stress
cargo run -- --nr-threads=4 -i 1000 --verbose --busy-timeout=0
```

I can produce a deadlock quite reliably.

With this change, I can't.

Even with 5 second busy timeout (the default), the run makes progress although it is slow as hell because of the busy timeout.
2025-10-16 22:00:01 +03:00
PThorpe92
9de9927b52 fix clippy warning 2025-10-16 14:47:40 -04:00
PThorpe92
04c9eee4f1 Throw parse error on GENERATED constraint when creating new table 2025-10-16 14:27:22 -04:00
PThorpe92
75e86b2c20 Throw parse error on GENERATED constraint in create table when opening new db 2025-10-16 14:26:58 -04:00
Preston Thorpe
b31908fe99 Merge 'translate/select: Fix rewriting Rowid expression when no btree table exists in joined table refs ' from Preston Thorpe
closes https://github.com/tursodatabase/turso/issues/3667

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3754
2025-10-16 14:22:51 -04:00
PThorpe92
e417188cb2 Fix panic when selecting explicit rowid from FROM clause subquery 2025-10-16 13:40:01 -04:00
PThorpe92
f033af4a93 Throw parse error on CHECK constraint in create table when opening new db 2025-10-16 13:30:05 -04:00
PThorpe92
bd33b3fa83 Throw parse error on CHECK constraint in create table 2025-10-16 13:07:12 -04:00
Pekka Enberg
6e34c505e1 Fix build after I/O many completions removal 2025-10-16 17:19:16 +03:00
Pekka Enberg
afa89c66c0 Merge 'Replace io_yield_many with completion groups' from Pekka Enberg
Reviewed-by: Pedro Muniz (@pedrocarlo)

Closes #3703
2025-10-16 17:17:43 +03:00
Pekka Enberg
08cd93caca Merge 'core/bree: remove duplicated code in BTreeCursor' from Pere Diaz Bou
Forgot to remove some functions from BTreeCursor moved to CursorTrait.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3751
2025-10-16 17:05:50 +03:00
Jussi Saurio
a1b6eafd03 Merge 'Fix: rolling back tx on Error should set autocommit to true' from Jussi Saurio
Rolling back a transaction on Error should result in
`connection.auto_commit` being set back to true.
Added a regression test for this where a UNIQUE constraint violation
rolls back the transaction and trying to COMMIT will fail.
Currently, our default conflict resolution strategy is ROLLBACK, which
ends the transaction. In SQLite, the default is ABORT, which rolls back
the current statement but allows the transaction to continue.
We should migrate to default ABORT once we support subtransactions.
Closes #3746

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3747
2025-10-16 16:55:54 +03:00
Pere Diaz Bou
57eb63cee0 core/bree: remove duplicated code in BTreeCursor 2025-10-16 14:50:08 +02:00
Preston Thorpe
1f70345dc1 Merge 'core: Unsafe Send and Sync pushdown' from Pekka Enberg
This patch pushes unsafe Send and Sync to individual components instead
of doing it at Database level. This makes it easier for us to
incrementally fix thread-safety, but avoid developers adding more thread
unsafe code.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3745
2025-10-16 08:19:45 -04:00
Preston Thorpe
7cc351afff Merge 'translate/insert: more refactoring and support INSERT OR IGNORE' from Preston Thorpe
`INSERT OR IGNORE INTO t VALUES (...)` can trivially be rewritten to
`INSERT INTO t VALUES (..) ON CONFLICT DO NOTHING`
This PR does this rewriting, as well as finishes a large refactor on
INSERT translation in general.. I just need a break from the rest of
this feature tbh.. just was getting under my skin and I have been in
`translate` land for too long.

Closes #3742
2025-10-16 08:18:28 -04:00
Jussi Saurio
e8e583ace6 Default ON CONFLICT behavior should be ROLLBACK 2025-10-16 14:28:18 +03:00
Jussi Saurio
56d5702176 Fix: rolling back tx should set autocommit to true
Rolling back a transaction should result in `connection.auto_commit` being set
back to true.

Added a regression test for this where a UNIQUE constraint violation rolls back
the transaction and trying to COMMIT will fail.

Currently, our default conflict resolution strategy is ROLLBACK,
which ends the transaction. In SQLite, the default is ABORT, which rolls back
the current statement but allows the transaction to continue.

We should migrate to default ABORT once we support subtransactions.
2025-10-16 12:05:53 +03:00
Pekka Enberg
bf5de920f2 core: Unsafe Send and Sync pushdown
This patch pushes unsafe Send and Sync to individual components instead
of doing it at Database level. This makes it easier for us to
incrementally fix thread-safety, but avoid developers adding more thread
unsafe code.
2025-10-16 11:26:50 +03:00
Pekka Enberg
b64ce77e47 Merge 'core: Don't run build.rs in debug mode' from Pedro Muniz
Hopefully this will help a bit with compile times when developing and
with `cargo check`

Closes #3744
2025-10-16 10:52:03 +03:00
Jussi Saurio
95f375791b refactor: move condition outside init_autoincrement 2025-10-16 09:34:13 +03:00
Jussi Saurio
25339a5200 rename: CheckConstraints -> ConstraintsToCheck
CHECK constraints is a separate SQL concept, so let's remove some
potential confusion from the naming.
2025-10-16 09:30:41 +03:00
pedrocarlo
2a1be48f3a do not run build.rs on debug mode 2025-10-16 01:22:54 -03:00
PThorpe92
41d2a0af77 Add INSERT OR IGNORE handling and refactor INSERT further 2025-10-15 22:51:10 -04:00
Preston Thorpe
4873103660 Merge 'Fix: outer CTEs should be available in subqueries' from Jussi Saurio
Closes #3670

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3737
2025-10-15 19:07:42 -04:00
Pekka Enberg
6fe9ea0925 Merge 'Make Rust bindings actually async' from Pedro Muniz
This PR introduces a `Context` object that is stored in the `Completion`
that currently only stores a `Waker`. In the future, I want to add some
sort of abort signal so that we can abort tasks that share the same
Context. To pass the Waker, I introduced a `step_with_waker` function in
`Statement` that delegates to an internal `_step` function. `_step` is
the previous `step` but just with the `Option<&Waker>` argument.
I was going to try and have the BusyHandler by truly async as well, but
I decided to not do it here, because it will be slightly complicated to
achieve.

Closes #3535
2025-10-15 19:38:24 +03:00
Jussi Saurio
d7a719418e Fix: outer CTEs should be available in subqueries 2025-10-15 15:15:55 +03:00
Pekka Enberg
af3a90bf4b core: Kill Many variant from IOCompletions enum 2025-10-15 11:48:24 +03:00
Pekka Enberg
5fb93b8780 core: Kill io_yield_many macro 2025-10-15 11:48:24 +03:00
Pekka Enberg
ff7f462981 core/vdbe/sorter: Replace IOCompletions::Many with completion group 2025-10-15 11:48:24 +03:00
Pekka Enberg
840d6a0df5 core/storage/btree: Replace io_yield_many with completion group in B-Tree 2025-10-15 11:48:24 +03:00
Pekka Enberg
986faa42da core/storage/pager: Replace io_yield_many with completion groups 2025-10-15 11:48:24 +03:00
Pekka Enberg
07ba7276b2 core/vdbe/sorter: Replace io_yield_many with completion groups 2025-10-15 11:48:24 +03:00
Pekka Enberg
80876148cf core/io: Add cancel() method to CompletionGroup 2025-10-15 11:48:24 +03:00
Pekka Enberg
7ff427f9d4 core/io: Fix completion group callback invocation for empty groups
Spotted by @pedrocarlo.
2025-10-15 11:48:24 +03:00
Pekka Enberg
cdd6f6a45d core/io: Fix completion group parent notification
The previous implementation of CompletionGroup would call the group's
callback function directly when the last completion finished:

    if prev == 1 {
        let group_result = group.result.get().and_then(|e| *e);
        (group.complete)(group_result.map_or(Ok(0), Err));
    }

This broke nested completion groups because parent groups track their
children via the Completion::callback() method. By calling the function
pointer directly, we bypassed the completion chain and parent groups
never received notification that their child had completed.

The fix stores a reference to the group's own Completion object in
self_completion during build(). When the last child finishes, we call
group_completion.callback() instead of invoking the function directly.
This properly propagates through the completion hierarchy, ensuring
parent groups decrement their outstanding count and eventually complete.

This matches the behavior of individual completions and maintains the
invariant that all completions notify their parents through the unified
callback() mechanism.
2025-10-15 10:47:16 +03:00
Pekka Enberg
d3e8285d93 core/io: Never skip a completion in CompletionGroup::add()
The previous implementation of CompletionGroup::add() would filter out
successfully-finished completions:

    if !completion.finished() || completion.failed() {
        self.completions.push(completion.clone());
    }

This caused a problem when combined with drain() in the calling code.
Completions that were already finished would be removed from the source
vector by drain() but not added to the group, effectively losing track
of them.

This breaks the invariant that all completions passed to a group must
be tracked, regardless of their state. The build() method already
handles finished completions correctly by not including them in the
outstanding count.

The fix is to always add all completions and let build() handle their
state appropriately, matching the behavior of the old io_yield_many!()
macro.
2025-10-15 10:47:16 +03:00
Jussi Saurio
4e6f373e3d Merge 'Fix: Evaluating expression in LIMIT and OFFSET clauses.' from
Closes #3687 .
Previously, the `try_fold_expr_to_i64` function casted `NULL` as `0`
when evaluating expressions in `LIMIT` or `OFFSET` clauses. I removed
this function since evaluating the expression directly and relying on
the MustBeInt operation for casting seems to handle everything.

Closes #3695
2025-10-15 10:36:36 +03:00
Jussi Saurio
bae33cb52c Avoid unwrapping failed f64 parsing attempts 2025-10-15 09:47:47 +03:00
Jussi Saurio
25cf56b8e8 Fix expected error message 2025-10-15 09:41:44 +03:00
Pekka Enberg
ab2911b370 Merge 'Fix change counter incrementation' from Jussi Saurio
We merged two concurrent fixes to `nchange` handling last night and
AFAICT the fix in #3692 was incorrect because it doesn't count UPDATEs
in cases where the original row was DELETEd as part of the UPDATE
statement.
The correct fix was in 87434b8
EDIT: okay, it's not strictly _incorrect_ in #3692 I guess, I just think
it's more intuitive to increment the change for UPDATE in the `insert`
opcode because that's what performs the actual update.

Closes #3735
2025-10-15 09:28:17 +03:00
Jussi Saurio
b1cb897216 Merge 'Fix another "should have been rewritten" translation panic' from Jussi Saurio
Closes #2158

Closes #3702
2025-10-15 09:25:01 +03:00
Jussi Saurio
2791f2f479 Fix change counter incrementation
We merged two concurrent fixes to `nchange` handling last night and
AFAICT the fix in #3692 was incorrect because it doesn't count UPDATEs
in cases where the original row was DELETEd as part of the UPDATE
statement.

The correct fix was in 87434b8
2025-10-15 08:51:27 +03:00