Current error messages are too "low level", e.g returning tokens in
messages. This PR improves this a bit.
Before:
```text
turso> with t as (select * from pragma_schema_version); select c.schema_version from t as c;
× unexpected token at SourceSpan { offset: SourceOffset(47), length: 1 }
╭────
1 │ with t as (select * from pragma_schema_version); select c.schema_version from t as c;
· ┬
· ╰── here
╰────
help: expected [TK_SELECT, TK_VALUES, TK_UPDATE, TK_DELETE, TK_INSERT, TK_REPLACE] but found TK_SEMI
```
Now:
```text
turso> with t as (select * from pragma_schema_version); select c.schema_version from t as c;
× unexpected token ';' at offset 47
╭────
1 │ with t as (select * from pragma_schema_version);select c.schema_version from t as c;
· ┬
· ╰── here
╰────
help: expected SELECT, VALUES, UPDATE, DELETE, INSERT, or REPLACE but found ';'
```
@TcMits WDYT?
Closes#3190
Depends on #3584 to use the most up-to-date implementation of
`ThreadRng`
- Add `fill_bytes` method to `IO`
- use `thread_rng` instead of `getrandom`, as `getrandom` is much slower
and `thread_rng` offers enough security
- modify `exec_randomblob`, `exec_random` and random_rowid generation to
use methods from IO for determinism
- modified simulator IO to implement `fill_bytes`
This the PRNG for sqlite if someone is curious. It is similar to
`thread_rng`:
```c
/* Initialize the state of the random number generator once,
** the first time this routine is called.
*/
if( wsdPrng.s[0]==0 ){
sqlite3_vfs *pVfs = sqlite3_vfs_find(0);
static const u32 chacha20_init[] = {
0x61707865, 0x3320646e, 0x79622d32, 0x6b206574
};
memcpy(&wsdPrng.s[0], chacha20_init, 16);
if( NEVER(pVfs==0) ){
memset(&wsdPrng.s[4], 0, 44);
}else{
sqlite3OsRandomness(pVfs, 44, (char*)&wsdPrng.s[4]);
}
wsdPrng.s[15] = wsdPrng.s[12];
wsdPrng.s[12] = 0;
wsdPrng.n = 0;
}
assert( N>0 );
while( 1 /* exit by break */ ){
if( N<=wsdPrng.n ){
memcpy(zBuf, &wsdPrng.out[wsdPrng.n-N], N);
wsdPrng.n -= N;
break;
}
if( wsdPrng.n>0 ){
memcpy(zBuf, wsdPrng.out, wsdPrng.n);
N -= wsdPrng.n;
zBuf += wsdPrng.n;
}
wsdPrng.s[12]++;
chacha_block((u32*)wsdPrng.out, wsdPrng.s);
wsdPrng.n = 64;
}
sqlite3_mutex_leave(mutex);
```
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#3799
The ProgrammingError exception is thrown when tables, indexes, or
columns are dropped in parallel. Let's not fail the Antithesis test
drivers when that happens.
Closes#3803
The ProgrammingError exception is thrown when tables, indexes, or
columns are dropped in parallel. Let's not fail the Antithesis test
drivers when that happens.
We have almost 1000 lines of code related to Completions the
`io/mod.rs`. My intention here is to declutter the file so that we can
focus more on what is relevant for the `IO` and `File` trait
Closes#3800
The collation fuzz test case takes up to 4 minutes to run, making it the
slowest of all the test cases. Let's reduce iteration count a bit to
make this more CI friendly.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#3797
The collation fuzz test case takes up to 4 minutes to run, making it the
slowest of all the test cases. Let's reduce iteration count a bit to
make this more CI friendly.
It was mentioned in https://github.com/tursodatabase/turso/pull/3720
that adding Miri support for `turso_stress` would be useful. And, that a
bash script to start Miri with the right config would be a big help.
Notable changes:
- `antithesis_sdk`'s default features are disabled at the workspace
level, and only enabled as needed with the `antithesis` feature flag in
the various turso crates. Miri needs the noop version of
`antithesis_sdk` to run `turso_stress`, and feature unification
previously prevented this. I'm not able to ensure locally that all the
Antithesis stuff is still happy with these changes.
- Bash script to run `turso_stress` - this is barebones for now, see
below
- Bash script to run `simulator` - this passes any args to the `cargo
run` invocation inside, intercepting `--seed` if it's present, and
generating one from `/dev/random` if it's not. The seed is passed to
both Miri and the simulator to keep the overall execution reproducible.
(I checked this with a simple case)
- A `const fn`, `normal_or_miri` to supply different defaults in things
like CLI args for normal operation and Miri, since it's so slow. (An
idea I stole from tokio.) Right now the relevant values are 100x smaller
for Miri, although Miri is probably 1000 to 10,000x slower overall from
a rough estimation.
Caught UB from running `turso_stress` with Miri:
- An unsafe cast of a `*u8` to `*u32` inside the BTree implementation
resulted in the `*u32` making an unaligned read: `read()` ->
`read_unaligned()` fixes this
Future work - Making `turso_stress` reproducible under Miri:
- Right now `turso_stress` is plugged in to Antithesis, which is great!
But, `antithesis_sdk`'s noop mode (`default-features = false`) turns
`antithesis_sdk::random::get_random()` into `rand::random<u64>()`, which
isn't seedable/reproducible. It's more work than I wanted to take on in
this PR, but I'd like to instead conditionally replace `get_random` with
a seedable `ChaCha8Rng` like in the simulator, if Miri is being used.
Comment:
- On a machine without all necessary dependencies, running the bash
scripts fails in a way that cargo prompts you through installing the
nightly toolchain, Miri, etc. until it works
- Below is a snippet of the output from Miri on the Btree alignment
issue. Because turso_stress isn't yet deterministic/reproducible under
Miri, I can't always reproduce it. (It doesn't always happen like the
ones in my last MR)
```
error: Undefined Behavior: accessing memory based on pointer with alignment 1, but alignment 4 is required
--> /home/rwp/git/turso/core/storage/btree.rs:2860:50
|
2860 | let mut pgno: u32 = unsafe { right_pointer.cast::<u32>().read().swap_bytes() };
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Undefined Behavior occurred here
|
= help: this indicates a bug in the program: it performed an invalid operation, and caused Undefined Behavior
= help: see https://doc.rust-lang.org/nightly/reference/behavior-considered-undefined.html for further information
```
Closes#3790
closes#3666
and probably other issues i'll have to go digging through to see if
there is any others.
<img width="948" height="445" alt="image" src="https://github.com/user-
attachments/assets/2844e09b-109a-4a70-bd18-d8a814e49ea0" />
Any ALTER COLUMN stmt will now update the constraints on the table
(primary key, foreign key, unique)
Closes#3776
After https://github.com/tursodatabase/turso/pull/3759 was merged, I
went back to the code to see if we could try and avoid this problem in
the future. One way I tried to achieve this is with scoped locking by
forcing all operations on the `SharedWalFile` to go through
`with_shared` and `with_shared_mut`. Also, I noticed some functions
still held locks across IO calls, so I fixed that as well.
If Rust already had`negative_impls` or custom auto traits in stable, I
could try to create a marker trait where you can mark any closure that
contains a `Completion` as IO related and throw a compile error when you
try to execute IO inside it.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#3781
