We have almost 1000 lines of code related to Completions the
`io/mod.rs`. My intention here is to declutter the file so that we can
focus more on what is relevant for the `IO` and `File` trait
Closes#3800
It was mentioned in https://github.com/tursodatabase/turso/pull/3720
that adding Miri support for `turso_stress` would be useful. And, that a
bash script to start Miri with the right config would be a big help.
Notable changes:
- `antithesis_sdk`'s default features are disabled at the workspace
level, and only enabled as needed with the `antithesis` feature flag in
the various turso crates. Miri needs the noop version of
`antithesis_sdk` to run `turso_stress`, and feature unification
previously prevented this. I'm not able to ensure locally that all the
Antithesis stuff is still happy with these changes.
- Bash script to run `turso_stress` - this is barebones for now, see
below
- Bash script to run `simulator` - this passes any args to the `cargo
run` invocation inside, intercepting `--seed` if it's present, and
generating one from `/dev/random` if it's not. The seed is passed to
both Miri and the simulator to keep the overall execution reproducible.
(I checked this with a simple case)
- A `const fn`, `normal_or_miri` to supply different defaults in things
like CLI args for normal operation and Miri, since it's so slow. (An
idea I stole from tokio.) Right now the relevant values are 100x smaller
for Miri, although Miri is probably 1000 to 10,000x slower overall from
a rough estimation.
Caught UB from running `turso_stress` with Miri:
- An unsafe cast of a `*u8` to `*u32` inside the BTree implementation
resulted in the `*u32` making an unaligned read: `read()` ->
`read_unaligned()` fixes this
Future work - Making `turso_stress` reproducible under Miri:
- Right now `turso_stress` is plugged in to Antithesis, which is great!
But, `antithesis_sdk`'s noop mode (`default-features = false`) turns
`antithesis_sdk::random::get_random()` into `rand::random<u64>()`, which
isn't seedable/reproducible. It's more work than I wanted to take on in
this PR, but I'd like to instead conditionally replace `get_random` with
a seedable `ChaCha8Rng` like in the simulator, if Miri is being used.
Comment:
- On a machine without all necessary dependencies, running the bash
scripts fails in a way that cargo prompts you through installing the
nightly toolchain, Miri, etc. until it works
- Below is a snippet of the output from Miri on the Btree alignment
issue. Because turso_stress isn't yet deterministic/reproducible under
Miri, I can't always reproduce it. (It doesn't always happen like the
ones in my last MR)
```
error: Undefined Behavior: accessing memory based on pointer with alignment 1, but alignment 4 is required
--> /home/rwp/git/turso/core/storage/btree.rs:2860:50
|
2860 | let mut pgno: u32 = unsafe { right_pointer.cast::<u32>().read().swap_bytes() };
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Undefined Behavior occurred here
|
= help: this indicates a bug in the program: it performed an invalid operation, and caused Undefined Behavior
= help: see https://doc.rust-lang.org/nightly/reference/behavior-considered-undefined.html for further information
```
Closes#3790
closes#3666
and probably other issues i'll have to go digging through to see if
there is any others.
<img width="948" height="445" alt="image" src="https://github.com/user-
attachments/assets/2844e09b-109a-4a70-bd18-d8a814e49ea0" />
Any ALTER COLUMN stmt will now update the constraints on the table
(primary key, foreign key, unique)
Closes#3776
DEFERRED was a bit too deferred - it allowed the dirty pages to be
written out to WAL before checking for violations, resulting in the
violations effectively being committed even though the transaction ended
up aborting
Closes#3784Closes#3785
DEFERRED was a bit too deferred - it allowed the dirty pages to be
written out to WAL before checking for violations, resulting in the
violations effectively being committed even though the transaction
ended up aborting
Without this change and running:
```
cd stress
while cargo run -- --nr-threads=4 -i 1000 --verbose --busy-timeout=0; do; done
```
I can produce a deadlock quite reliably.
With this change, I can't.
Even with 5 second busy timeout (the default), the run makes progress
although it is slow as hell because of the busy timeout.
Full disclosure: i couldn't figure out based on parking lot RwLock
semantics why this would fix it, so maybe it just lessens the
probability
Reviewed-by: Preston Thorpe <preston@turso.tech>
Closes#3759
Without this change and running:
```
cd stress
cargo run -- --nr-threads=4 -i 1000 --verbose --busy-timeout=0
```
I can produce a deadlock quite reliably.
With this change, I can't.
Even with 5 second busy timeout (the default), the run makes progress although it is slow as hell because of the busy timeout.
Rolling back a transaction on Error should result in
`connection.auto_commit` being set back to true.
Added a regression test for this where a UNIQUE constraint violation
rolls back the transaction and trying to COMMIT will fail.
Currently, our default conflict resolution strategy is ROLLBACK, which
ends the transaction. In SQLite, the default is ABORT, which rolls back
the current statement but allows the transaction to continue.
We should migrate to default ABORT once we support subtransactions.
Closes#3746
Reviewed-by: Preston Thorpe <preston@turso.tech>
Closes#3747
This patch pushes unsafe Send and Sync to individual components instead
of doing it at Database level. This makes it easier for us to
incrementally fix thread-safety, but avoid developers adding more thread
unsafe code.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#3745
`INSERT OR IGNORE INTO t VALUES (...)` can trivially be rewritten to
`INSERT INTO t VALUES (..) ON CONFLICT DO NOTHING`
This PR does this rewriting, as well as finishes a large refactor on
INSERT translation in general.. I just need a break from the rest of
this feature tbh.. just was getting under my skin and I have been in
`translate` land for too long.
Closes#3742
Rolling back a transaction should result in `connection.auto_commit` being set
back to true.
Added a regression test for this where a UNIQUE constraint violation rolls back
the transaction and trying to COMMIT will fail.
Currently, our default conflict resolution strategy is ROLLBACK,
which ends the transaction. In SQLite, the default is ABORT, which rolls back
the current statement but allows the transaction to continue.
We should migrate to default ABORT once we support subtransactions.
This patch pushes unsafe Send and Sync to individual components instead
of doing it at Database level. This makes it easier for us to
incrementally fix thread-safety, but avoid developers adding more thread
unsafe code.
