Commit Graph

10433 Commits

Author SHA1 Message Date
Jussi Saurio
2a03c1a617 Add subjournal and savepoints to Pager struct 2025-10-22 23:40:44 +03:00
Jussi Saurio
8b15a06a85 Add Savepoint struct 2025-10-22 23:40:44 +03:00
Jussi Saurio
459c01f93c Add subjournal module
The subjournal is a temporary file where stmt subtransactions write an
'undo log' of pages before modifying them. If a stmt subtransaction
rolls back, the pages are restored from the subjournal.
2025-10-22 23:40:44 +03:00
Jussi Saurio
ad80285437 Rename is_scope to deferred and invert respective boolean logic
Much clearer name for what it is/does
2025-10-22 23:40:44 +03:00
Jussi Saurio
d4a9797f79 Store two foreign key counters in ProgramState
1. The number of deferred FK violations when the statement started.
   When a statement subtransaction rolls back, the connection's
   deferred violation counter will be reset to this value.
2. The number of immediate FK violations that occurred during the
   statement. In practice we just need to know whether this number
   is nonzero, and if it is, the statement subtransaction will roll
   back.

Statement subtransactions will be implemented in future commits.
2025-10-22 23:40:44 +03:00
Jussi Saurio
6557a41503 Refactor emit_fk_violation() to always issue a FkCounter instruction 2025-10-22 23:40:44 +03:00
Pekka Enberg
7b6667d079 Merge 'Add AtomicEnum proc macro to generate atomic wrappers to replace RwLocks' from Preston Thorpe
This PR adds the following derive macro
`AtomicEnum`
for the cases like the following:
```rust
pub enum SyncMode {
    Off = 0,
    Full = 2,
}
// or
pub enum CipherMode {
    Aes128Gcm,
    Aes256Gcm,
    Aegis256,
    Aegis128L,
    Aegis128X2,
    Aegis128X4,
    Aegis256X2,
    Aegis256X4,
}
```
Which are very basic enums, but which currently either require a
`RwLock` (the current solution for both of the above), or they require a
hand rolled atomic wrapper to keep the state without the lock.
```rust
pub struct AtomicDbState(AtomicUsize);

impl AtomicDbState {
    #[inline]
    pub const fn new(state: DbState) -> Self {
        Self(AtomicUsize::new(state as usize))
    }
```
This PR adds `AtomicEnum` derive macro which generates and let's us use
`AtomicDbState` or `AtomicCipherMode`, and derives `get`, `set` and
`swap` methods on them.
Each enum can have up to 1 named or unnamed field, and it supports i8/u8
and boolean types, which it encodes into half of a u16, with the
discriminant in the other half. Otherwise, it will just use a u8 and
encode the boolean into the 7th bit.

Closes #3766
2025-10-22 17:07:58 +03:00
PThorpe92
a8b257c664 Replace several RwLock<Enum> values with new AtomicEnums 2025-10-22 09:35:26 -04:00
PThorpe92
123a26b7d9 Add AtomicEnum macro macros crate 2025-10-22 09:27:14 -04:00
Pekka Enberg
b984ddf98f Turso 0.3.0-pre.4 2025-10-22 13:42:52 +03:00
Pekka Enberg
4ac45e0d75 Merge 'github: Run fuzz tests in a separate workflow' from Pekka Enberg
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3807
2025-10-22 13:42:32 +03:00
Pekka Enberg
c944bd312e Merge 'tests: Separate integration and fuzz tests' from Pekka Enberg
This separates fuzz tests from integration tests so that you can run the
fast test cases with:
```
cargo test --test integration_tests
```
and the longer fuzz cases with:
```
cargo test --test fuzz_tests
```

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3806
2025-10-22 13:42:23 +03:00
Pekka Enberg
da828681c3 github: Run fuzz tests in a separate workflow 2025-10-22 13:13:59 +03:00
Pekka Enberg
29b400c6ac tests: Separate integration and fuzz tests
This separates fuzz tests from integration tests so that you can run the
fast test cases with:

```
cargo test --test integration_tests
```

and the longer fuzz cases with:

```
cargo test --test fuzz_tests
```
2025-10-22 13:05:29 +03:00
Pekka Enberg
1aba105df4 Merge 'Vector speedup' from Nikita Sivukhin
This PR reduces allocation for vector distance calculation and also use
[simsimd](https://github.com/ashvardanian/SimSIMD) library to execute
cosine/l2 distances for dense vectors.

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #3802
2025-10-22 12:28:43 +03:00
Jussi Saurio
ffc601b4b0 Merge 'Return better syntax error messages' from Diego Reis
Current error messages are too "low level", e.g returning tokens in
messages. This PR improves this a bit.
Before:
```text
 turso> with t as (select * from pragma_schema_version); select c.schema_version from t as c;

  × unexpected token at SourceSpan { offset: SourceOffset(47), length: 1 }
   ╭────
 1 │ with t as (select * from pragma_schema_version); select c.schema_version from t as c;
   ·                                                ┬
   ·                                                ╰── here
   ╰────
  help: expected [TK_SELECT, TK_VALUES, TK_UPDATE, TK_DELETE, TK_INSERT, TK_REPLACE] but found TK_SEMI
```
Now:
```text
 turso> with t as (select * from pragma_schema_version); select c.schema_version from t as c;

  × unexpected token ';' at offset 47
   ╭────
 1 │ with t as (select * from pragma_schema_version);select c.schema_version from t as c;
   ·                                                ┬
   ·                                                ╰── here
   ╰────
  help: expected SELECT, VALUES, UPDATE, DELETE, INSERT, or REPLACE but found ';'
  ```
@TcMits WDYT?

Closes #3190
2025-10-22 10:57:54 +03:00
Nikita Sivukhin
7d423d358f avoid unnecessary time measures 2025-10-22 11:08:46 +04:00
Pekka Enberg
d45a8da2f2 Merge 'parser: translate boolean values to literal when parsing column constraints' from Preston Thorpe
closes #3796

Closes #3804
2025-10-22 09:11:03 +03:00
Pekka Enberg
1aad1b224a Merge 'core/io: Make random generation deterministically simulated' from Pedro Muniz
Depends on #3584 to use the most up-to-date implementation of
`ThreadRng`
- Add `fill_bytes` method to `IO`
- use `thread_rng` instead of `getrandom`, as `getrandom` is much slower
and `thread_rng` offers enough security
- modify `exec_randomblob`, `exec_random` and random_rowid generation to
use methods from IO for determinism
- modified simulator IO to implement `fill_bytes`
This the PRNG for sqlite if someone is curious. It is similar to
`thread_rng`:
```c
/* Initialize the state of the random number generator once,
  ** the first time this routine is called.
  */
  if( wsdPrng.s[0]==0 ){
    sqlite3_vfs *pVfs = sqlite3_vfs_find(0);
    static const u32 chacha20_init[] = {
      0x61707865, 0x3320646e, 0x79622d32, 0x6b206574
    };
    memcpy(&wsdPrng.s[0], chacha20_init, 16);
    if( NEVER(pVfs==0) ){
      memset(&wsdPrng.s[4], 0, 44);
    }else{
      sqlite3OsRandomness(pVfs, 44, (char*)&wsdPrng.s[4]);
    }
    wsdPrng.s[15] = wsdPrng.s[12];
    wsdPrng.s[12] = 0;
    wsdPrng.n = 0;
  }

  assert( N>0 );
  while( 1 /* exit by break */ ){
    if( N<=wsdPrng.n ){
      memcpy(zBuf, &wsdPrng.out[wsdPrng.n-N], N);
      wsdPrng.n -= N;
      break;
    }
    if( wsdPrng.n>0 ){
      memcpy(zBuf, wsdPrng.out, wsdPrng.n);
      N -= wsdPrng.n;
      zBuf += wsdPrng.n;
    }
    wsdPrng.s[12]++;
    chacha_block((u32*)wsdPrng.out, wsdPrng.s);
    wsdPrng.n = 64;
  }
  sqlite3_mutex_leave(mutex);
```

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #3799
2025-10-22 09:10:36 +03:00
Pekka Enberg
9005458aa2 Merge 'antithesis-tests: Don't fail tests on ProgrammingError' from Pekka Enberg
The ProgrammingError exception is thrown when tables, indexes, or
columns are dropped in parallel. Let's not fail the Antithesis test
drivers when that happens.

Closes #3803
2025-10-22 09:10:30 +03:00
PThorpe92
2f401c0bcc Add regression tcl test for #3796 default bool col constraints 2025-10-21 21:22:09 -04:00
PThorpe92
892fcc881d Handle TRUE|FALSE literal case for default column constraint in the parser 2025-10-21 21:15:35 -04:00
Pere Diaz Bou
3227caaa1d Merge 'core: move BTreeCursor under MVCC cursor' from Pere Diaz Bou
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3756
2025-10-21 19:20:49 +02:00
Nikita Sivukhin
10ead9f3b6 one more clippy fix 2025-10-21 21:10:58 +04:00
pedrocarlo
72baf48863 add random generation in simulator IO 2025-10-21 14:10:38 -03:00
pedrocarlo
8c0b9c6979 add additional fill_bytes method to IO to deterministically generate
random bytes and modify random functions to use them
2025-10-21 14:10:38 -03:00
pedrocarlo
8501bc930a use workspace rand version 2025-10-21 14:10:05 -03:00
Pekka Enberg
4805c6b06b antithesis-tests: Don't fail tests on ProgrammingError
The ProgrammingError exception is thrown when tables, indexes, or
columns are dropped in parallel. Let's not fail the Antithesis test
drivers when that happens.
2025-10-21 20:08:11 +03:00
Pekka Enberg
eb835c39d4 Merge 'core/vdbe: fix ALTER COLUMN to propagate constraints to other table references' from Preston Thorpe
with fix to ensure `PRIMARY KEY` isn't written twice

Closes #3798
2025-10-21 20:05:41 +03:00
Nikita Sivukhin
792e0033ae fix tests and clippy 2025-10-21 21:03:45 +04:00
Pekka Enberg
177e4f39a8 Merge 'Move completion code to separate file' from Pedro Muniz
We have almost 1000 lines of code related to Completions the
`io/mod.rs`. My intention here is to declutter the file so that we can
focus more on what is relevant for the `IO` and `File` trait

Closes #3800
2025-10-21 20:03:40 +03:00
Pekka Enberg
edea108037 Merge 'avoid unnecessary allocations' from Nikita Sivukhin
Closes #3801
2025-10-21 20:03:25 +03:00
Pere Diaz Bou
128b426681 core/mvcc/cursor: imports 2025-10-21 18:22:37 +02:00
Pere Diaz Bou
92c0e74458 core/mvcc/cursor: implement seek_to_last 2025-10-21 18:22:37 +02:00
Pere Diaz Bou
8857604161 core/mvcc/cursor: fix rewind 2025-10-21 18:22:37 +02:00
Pere Diaz Bou
3f41a092f2 core/mvcc/cursor: add next rowid lock 2025-10-21 18:22:37 +02:00
Pere Diaz Bou
0fee588bca core/mvcc/cursor: add record cursor 2025-10-21 18:22:37 +02:00
Pere Diaz Bou
790859c62f core/mvcc/cursor: fix exists 2025-10-21 18:22:37 +02:00
Pere Diaz Bou
edac1ff256 core/mvcc/cursor: set null flag 2025-10-21 18:22:37 +02:00
Pere Diaz Bou
ea04e9033a core/mvcc: add btree_cursor under MVCC cursor 2025-10-21 18:22:37 +02:00
Nikita Sivukhin
00e382a7c7 avoid unnecessary allocations 2025-10-21 20:13:39 +04:00
Pekka Enberg
d2d995a9c0 Merge 'Make sure explicit column aliases have binding precedence in orderby' from Pavan Nambi
closes https://github.com/tursodatabase/turso/issues/3684

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3709
2025-10-21 19:04:42 +03:00
pedrocarlo
a327747531 organize completion code in a separate file 2025-10-21 12:43:49 -03:00
Pekka Enberg
51c5d6a66c Merge 'tests/integration: Reduce collation fuzz test iterations' from Pekka Enberg
The collation fuzz test case takes up to 4 minutes to run, making it the
slowest of all the test cases. Let's reduce iteration count a bit to
make this more CI friendly.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3797
2025-10-21 18:41:10 +03:00
PThorpe92
08197e345a Fix cdc test to assert for correct schema output 2025-10-21 11:22:29 -04:00
PThorpe92
c48d7a0963 Add tcl tests for alter column fixes 2025-10-21 10:47:08 -04:00
PThorpe92
2fbd4b7cec Ensure op_alter_column and Func::AlterColumn are fixing table references to columns with fk's 2025-10-21 10:46:52 -04:00
PThorpe92
06e3b9611b Add helpers to rewrite REFERENCES from foriegn keys in ColumnDefinition 2025-10-21 10:40:31 -04:00
Pekka Enberg
05bd75275f tests/integration: Reduce collation fuzz test iterations
The collation fuzz test case takes up to 4 minutes to run, making it the
slowest of all the test cases. Let's reduce iteration count a bit to
make this more CI friendly.
2025-10-21 17:29:11 +03:00
PThorpe92
7c746e476c Fix to_sql method on BTreeTable to not double write primary keys 2025-10-21 09:45:42 -04:00