Commit Graph

10360 Commits

Author SHA1 Message Date
Pekka Enberg
f4da2194f4 Merge 'Shared WAL lock scoping' from Pedro Muniz
After https://github.com/tursodatabase/turso/pull/3759 was merged, I
went back to the code to see if we could try and avoid this problem in
the future. One way I tried to achieve this is with scoped locking by
forcing all operations on the `SharedWalFile` to go through
`with_shared` and `with_shared_mut`. Also, I noticed some functions
still held locks across IO calls, so I fixed that as well.
If Rust already had`negative_impls` or custom auto traits in stable, I
could try to create a marker trait where you can mark any closure that
contains a `Completion` as IO related and throw a compile error when you
try to execute IO inside it.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3781
2025-10-21 09:05:40 +03:00
pedrocarlo
ba9a1ebbef add mutable scoped locking for SharedWalFile 2025-10-20 10:45:14 -03:00
pedrocarlo
b00a276960 add scoped locking for SharedWalFile to avoid holding locks for longer than needed 2025-10-20 10:45:14 -03:00
Pekka Enberg
97991a1934 Merge 'Fix deferred FK violations check before committing to WAL' from Jussi Saurio
DEFERRED was a bit too deferred - it allowed the dirty pages to be
written out to WAL before checking for violations, resulting in the
violations effectively being committed even though the transaction ended
up aborting
Closes #3784

Closes #3785
2025-10-20 14:51:25 +03:00
Jussi Saurio
10532544dc Fix: check deferred FK violations before committing to WAL
DEFERRED was a bit too deferred - it allowed the dirty pages to be
written out to WAL before checking for violations, resulting in the
violations effectively being committed even though the transaction
ended up aborting
2025-10-20 14:00:49 +03:00
Jussi Saurio
bebe230b05 Regression test: deferred FK violations are checked before commit 2025-10-20 13:59:37 +03:00
Pekka Enberg
c81c062707 Merge 'Remove tests that alter testing.db from views.test' from Preston Thorpe
so we don't to have to `git restore` the database files every time tests
run

Closes #3772
2025-10-20 13:43:56 +03:00
Pekka Enberg
931a2a4127 Merge 'tests/integration: Disable rowid alias differential fuzz test case' from Pekka Enberg
The fuzz test seems to find something that causes the tests to hang.
Let's disable it for now.

Closes #3782
2025-10-20 13:38:57 +03:00
Pekka Enberg
591b43634e tests/integration: Disable rowid alias differential fuzz test case
The fuzz test seems to find something that causes the tests to hang.
Let's disable it for now.
2025-10-20 12:30:32 +03:00
Pekka Enberg
e6c70f2d55 Merge 'core/storage: Reduce logging level' from Pekka Enberg
Closes #3765
2025-10-20 11:52:53 +03:00
Pekka Enberg
583f2b3d48 Merge 'cli: Improve manual page display' from Pavan Nambi
opens manual page in seperate screen similar to man command in linux

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3777
2025-10-20 11:51:59 +03:00
Pavan-Nambi
aad3c00e57 clippy 2025-10-19 13:25:40 +05:30
Pavan-Nambi
055b38787d more keybindings 2025-10-19 13:22:35 +05:30
Pavan-Nambi
3658a94f06 fmt 2025-10-19 13:20:40 +05:30
Pavan-Nambi
ba2570d0b8 cli:scrolling and enable suggestion for wrong commands 2025-10-19 13:20:03 +05:30
PThorpe92
b837232b13 Remove tests that alter testing.db from views.test 2025-10-18 12:05:33 -04:00
Pekka Enberg
e03f6dbf94 core/storage: Reduce logging level 2025-10-17 20:09:00 +03:00
Pekka Enberg
00e6ed82b6 Merge 'tests: Add rowid alias fuzz test case' from Pekka Enberg
This adds a new fuzz test case to verify that any query returns the same
results with and without a rowid alias.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2952
2025-10-17 14:30:18 +03:00
Pekka Enberg
3020966fbd Turso 0.3.0-pre.3 2025-10-17 11:08:36 +03:00
Pekka Enberg
6f71059f07 tests: Add rowid alias fuzz test case
This adds a new fuzz test case to verify that any query returns the same
results with and without a rowid alias.
2025-10-17 10:47:07 +03:00
Pekka Enberg
0fcb0b889a sql_generation: Fix predicate evaluation for JOIN operations 2025-10-17 10:47:07 +03:00
Pekka Enberg
e3ec4f7ea2 sql_generation: Add support for predefined columns 2025-10-17 10:47:07 +03:00
Pekka Enberg
2d05d062b2 Merge 'antithesis-tests: don't create CHECK constraints' from Jussi Saurio
we don't support them, and we just started returning parse errors for
trying to do so -> failures in antithesis runs

Closes #3763
2025-10-17 10:46:23 +03:00
Pekka Enberg
62fff37ea6 Merge 'stress: prevent thread from holding write lock and then stopping' from Jussi Saurio
When a stress thread runs out of work, execute COMMIT and ignore the
result.
This prevents the currently extremely common scenario where:
- Thread x does BEGIN
- Thread x does e.g. INSERT ...
Then runs out of iterations and stops. No other threads can write
anything and they just wait for 5 seconds (busy timeout) and then try
again, forever.
Closes #3697

Closes #3762
2025-10-17 09:36:51 +03:00
Jussi Saurio
d187be74a5 antithesis-tests: don't create CHECK constraints
we don't support them, and we just started returning parse errors for
trying to do so -> failures in antithesis runs
2025-10-17 09:25:06 +03:00
Jussi Saurio
56e9ba9eae Merge 'WAL: don't hold shared lock across IO operations' from Jussi Saurio
Without this change and running:
```
cd stress
while cargo run -- --nr-threads=4 -i 1000 --verbose --busy-timeout=0; do; done
```
I can produce a deadlock quite reliably.
With this change, I can't.
Even with 5 second busy timeout (the default), the run makes progress
although it is slow as hell because of the busy timeout.
Full disclosure: i couldn't figure out based on parking lot RwLock
semantics why this would fix it, so maybe it just lessens the
probability

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3759
2025-10-17 08:50:26 +03:00
Jussi Saurio
bae3a42564 stress: prevent thread from holding write lock and then stopping
When a stress thread runs out of work, execute COMMIT and ignore
the result.

This prevents the currently extremely common scenario where:

- Thread x does BEGIN
- Thread x does e.g. INSERT ...

Then runs out of iterations and stops. No other threads can write
anything and they just wait for 5 seconds (busy timeout) and then
try again, forever.
2025-10-17 08:47:40 +03:00
Preston Thorpe
a82e8c9c23 Merge 'translate/select: prevent multiple identical non-aliased table references ' from Preston Thorpe
closes https://github.com/tursodatabase/turso/issues/3505

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3761
2025-10-16 19:03:23 -04:00
PThorpe92
ddd674c340 Move duplicate table identifier checking to parse_join to allow for natural joins 2025-10-16 18:32:48 -04:00
PThorpe92
79c5234122 Add TCL test for self ambiguous join 2025-10-16 16:43:08 -04:00
PThorpe92
10c69b910e Prevent ambiguous self-join table reference 2025-10-16 16:39:10 -04:00
Preston Thorpe
119984d98b Merge 'Prevent on conflict column definitions on CREATE TABLE or opening DB' from Preston Thorpe
more parse errors!

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3760
2025-10-16 16:22:40 -04:00
PThorpe92
edaa1b675e Prevent column definitions on CREATE TABLE or opening DB with ON CONFLICT on column def 2025-10-16 15:45:20 -04:00
Preston Thorpe
563c67dfea Merge 'core/translate: throw parse error on unsupported GENERATED column constraints' from Preston Thorpe
similar to #3755, we do this for #3749

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3758
2025-10-16 15:41:39 -04:00
Jussi Saurio
2ca388d78d WAL: don't hold shared lock across IO operations
Without this change and running:

```
cd stress
cargo run -- --nr-threads=4 -i 1000 --verbose --busy-timeout=0
```

I can produce a deadlock quite reliably.

With this change, I can't.

Even with 5 second busy timeout (the default), the run makes progress although it is slow as hell because of the busy timeout.
2025-10-16 22:00:01 +03:00
PThorpe92
9de9927b52 fix clippy warning 2025-10-16 14:47:40 -04:00
PThorpe92
04c9eee4f1 Throw parse error on GENERATED constraint when creating new table 2025-10-16 14:27:22 -04:00
PThorpe92
75e86b2c20 Throw parse error on GENERATED constraint in create table when opening new db 2025-10-16 14:26:58 -04:00
Preston Thorpe
b31908fe99 Merge 'translate/select: Fix rewriting Rowid expression when no btree table exists in joined table refs ' from Preston Thorpe
closes https://github.com/tursodatabase/turso/issues/3667

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3754
2025-10-16 14:22:51 -04:00
Preston Thorpe
06dc2b5376 Merge 'Throw parse error on CHECK constraint in create table' from Preston Thorpe
RE: https://github.com/tursodatabase/turso/issues/3753
Let's throw parse error until this feature is supported.
Should we throw parse error if we open a DB up that has CHECK
constraints on the columns?

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3755
2025-10-16 14:22:21 -04:00
PThorpe92
959df4c4bc Add TCL test for rowid from clause subquery panic 2025-10-16 13:40:10 -04:00
PThorpe92
e417188cb2 Fix panic when selecting explicit rowid from FROM clause subquery 2025-10-16 13:40:01 -04:00
PThorpe92
f033af4a93 Throw parse error on CHECK constraint in create table when opening new db 2025-10-16 13:30:05 -04:00
PThorpe92
bd33b3fa83 Throw parse error on CHECK constraint in create table 2025-10-16 13:07:12 -04:00
Pekka Enberg
9d4813df04 Reduce Antithesis runtime to 4 hours 2025-10-16 19:00:40 +03:00
Preston Thorpe
79e4176163 Merge 'cli: .tables and .indexes to show data from attached tables aswell' from Konstantinos Artopoulos
Closes #3545

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3627
2025-10-16 11:42:29 -04:00
Pekka Enberg
6e34c505e1 Fix build after I/O many completions removal 2025-10-16 17:19:16 +03:00
Jussi Saurio
44fa0a4f6e Merge 'bindings/rust: propagate the DropBehavior of a dropped tx to next access of DB, instead of panicing' from Jussi Saurio
Closes #3748
Right now if any error happens during an interactive tx that causes the
`Transaction` to drop, the program will panic.
To prevent this, we store the `DropBehavior` of the transaction on the
`Connection` when it drops and issue the corresponding action (ROLLBACK
/ COMMIT / IGNORE / PANIC) the next time `Connection` is used to access
the database. This defaults to `IGNORE`.
I don't know how good this solution is, but we can at least prevent a
panic by storing whether the connection has a dangling transaction and
roll it back automatically the next time the connection tries to do
something.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3750
2025-10-16 17:18:31 +03:00
Pekka Enberg
afa89c66c0 Merge 'Replace io_yield_many with completion groups' from Pekka Enberg
Reviewed-by: Pedro Muniz (@pedrocarlo)

Closes #3703
2025-10-16 17:17:43 +03:00
Pekka Enberg
08cd93caca Merge 'core/bree: remove duplicated code in BTreeCursor' from Pere Diaz Bou
Forgot to remove some functions from BTreeCursor moved to CursorTrait.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3751
2025-10-16 17:05:50 +03:00