Commit Graph

10455 Commits

Author SHA1 Message Date
Jussi Saurio
a14bbdecf2 Add assertion that page is loaded when pager.add_dirty() is called 2025-10-22 23:40:45 +03:00
Jussi Saurio
7376475cb3 Do not start statement subtransactions when MVCC is enabled
MVCC does not support statement-level rollback.
2025-10-22 23:40:45 +03:00
Jussi Saurio
e9bfb57065 Fix incorrectly implemented test
Test started executing another statement when previous statement
returned IO the last time and didn't run to completion
2025-10-22 23:40:45 +03:00
Jussi Saurio
1dcfd3d068 fix stale test: constraint errors do not roll back tx anymore 2025-10-22 23:40:45 +03:00
Jussi Saurio
d8cc57cf14 clippy: Remove unnecessary referencing 2025-10-22 23:40:45 +03:00
Jussi Saurio
2d3ac79fe9 Modify fk_deferred_constraints_fuzz
- Add more statements per iteration
- Allow interactive transaction to contain multiple statements
- add VERBOSE flag to print all statements executed in a successful
  iteration
2025-10-22 23:40:45 +03:00
Jussi Saurio
1fdc0258cd Unignore fk_deferred_constraints_fuzz because it doesnt fail anymore 2025-10-22 23:40:45 +03:00
Jussi Saurio
97aad78b3f Allow dead code - SQLITE_CONSTRAINT_FOREIGNKEY is currently unused 2025-10-22 23:40:45 +03:00
Jussi Saurio
086ba8c946 VDBE: begin statement subtransaction in op_transaction 2025-10-22 23:40:45 +03:00
Jussi Saurio
904cbe535d VDBE: handle subtransaction commits/aborts in op_halt 2025-10-22 23:40:45 +03:00
Jussi Saurio
f0548c280f ProgramState: add begin_statement() and end_statement() 2025-10-22 23:40:45 +03:00
Jussi Saurio
734eeb5bab VDBE: constraint errors do not cause a tx rollback by default 2025-10-22 23:40:45 +03:00
Jussi Saurio
25f8ba0025 Pager: clear savepoints when tx rolls back 2025-10-22 23:40:45 +03:00
Jussi Saurio
a8cf8e4594 Pager: subjournal page if required when it's marked as dirty 2025-10-22 23:40:45 +03:00
Jussi Saurio
97177dae02 add missing imports 2025-10-22 23:40:44 +03:00
Jussi Saurio
f4af7c2242 Pager: add begin_statement() method 2025-10-22 23:40:44 +03:00
Jussi Saurio
a19c5c22ac Pager: add rollback_to_newest_savepoint() method 2025-10-22 23:40:44 +03:00
Jussi Saurio
86d5ad6815 pager: allow upserted cached page not to be dirty 2025-10-22 23:40:44 +03:00
Jussi Saurio
5b01605fae Pager: add subjournal_page_if_required() method 2025-10-22 23:40:44 +03:00
Jussi Saurio
e8226c0e4b Pager: add clear_savepoint() method 2025-10-22 23:40:44 +03:00
Jussi Saurio
aa1eebbfcb Pager: add open_savepoint() and release_savepoint() methods 2025-10-22 23:40:44 +03:00
Jussi Saurio
77be1f08ae Pager: add open_subjournal method 2025-10-22 23:40:44 +03:00
Jussi Saurio
2a03c1a617 Add subjournal and savepoints to Pager struct 2025-10-22 23:40:44 +03:00
Jussi Saurio
8b15a06a85 Add Savepoint struct 2025-10-22 23:40:44 +03:00
Jussi Saurio
459c01f93c Add subjournal module
The subjournal is a temporary file where stmt subtransactions write an
'undo log' of pages before modifying them. If a stmt subtransaction
rolls back, the pages are restored from the subjournal.
2025-10-22 23:40:44 +03:00
Jussi Saurio
ad80285437 Rename is_scope to deferred and invert respective boolean logic
Much clearer name for what it is/does
2025-10-22 23:40:44 +03:00
Jussi Saurio
d4a9797f79 Store two foreign key counters in ProgramState
1. The number of deferred FK violations when the statement started.
   When a statement subtransaction rolls back, the connection's
   deferred violation counter will be reset to this value.
2. The number of immediate FK violations that occurred during the
   statement. In practice we just need to know whether this number
   is nonzero, and if it is, the statement subtransaction will roll
   back.

Statement subtransactions will be implemented in future commits.
2025-10-22 23:40:44 +03:00
Jussi Saurio
6557a41503 Refactor emit_fk_violation() to always issue a FkCounter instruction 2025-10-22 23:40:44 +03:00
Pekka Enberg
7b6667d079 Merge 'Add AtomicEnum proc macro to generate atomic wrappers to replace RwLocks' from Preston Thorpe
This PR adds the following derive macro
`AtomicEnum`
for the cases like the following:
```rust
pub enum SyncMode {
    Off = 0,
    Full = 2,
}
// or
pub enum CipherMode {
    Aes128Gcm,
    Aes256Gcm,
    Aegis256,
    Aegis128L,
    Aegis128X2,
    Aegis128X4,
    Aegis256X2,
    Aegis256X4,
}
```
Which are very basic enums, but which currently either require a
`RwLock` (the current solution for both of the above), or they require a
hand rolled atomic wrapper to keep the state without the lock.
```rust
pub struct AtomicDbState(AtomicUsize);

impl AtomicDbState {
    #[inline]
    pub const fn new(state: DbState) -> Self {
        Self(AtomicUsize::new(state as usize))
    }
```
This PR adds `AtomicEnum` derive macro which generates and let's us use
`AtomicDbState` or `AtomicCipherMode`, and derives `get`, `set` and
`swap` methods on them.
Each enum can have up to 1 named or unnamed field, and it supports i8/u8
and boolean types, which it encodes into half of a u16, with the
discriminant in the other half. Otherwise, it will just use a u8 and
encode the boolean into the 7th bit.

Closes #3766
2025-10-22 17:07:58 +03:00
PThorpe92
a8b257c664 Replace several RwLock<Enum> values with new AtomicEnums 2025-10-22 09:35:26 -04:00
PThorpe92
123a26b7d9 Add AtomicEnum macro macros crate 2025-10-22 09:27:14 -04:00
Pekka Enberg
b984ddf98f Turso 0.3.0-pre.4 2025-10-22 13:42:52 +03:00
Pekka Enberg
4ac45e0d75 Merge 'github: Run fuzz tests in a separate workflow' from Pekka Enberg
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3807
2025-10-22 13:42:32 +03:00
Pekka Enberg
c944bd312e Merge 'tests: Separate integration and fuzz tests' from Pekka Enberg
This separates fuzz tests from integration tests so that you can run the
fast test cases with:
```
cargo test --test integration_tests
```
and the longer fuzz cases with:
```
cargo test --test fuzz_tests
```

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3806
2025-10-22 13:42:23 +03:00
Pekka Enberg
da828681c3 github: Run fuzz tests in a separate workflow 2025-10-22 13:13:59 +03:00
Pekka Enberg
29b400c6ac tests: Separate integration and fuzz tests
This separates fuzz tests from integration tests so that you can run the
fast test cases with:

```
cargo test --test integration_tests
```

and the longer fuzz cases with:

```
cargo test --test fuzz_tests
```
2025-10-22 13:05:29 +03:00
Pekka Enberg
1aba105df4 Merge 'Vector speedup' from Nikita Sivukhin
This PR reduces allocation for vector distance calculation and also use
[simsimd](https://github.com/ashvardanian/SimSIMD) library to execute
cosine/l2 distances for dense vectors.

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #3802
2025-10-22 12:28:43 +03:00
Jussi Saurio
ffc601b4b0 Merge 'Return better syntax error messages' from Diego Reis
Current error messages are too "low level", e.g returning tokens in
messages. This PR improves this a bit.
Before:
```text
 turso> with t as (select * from pragma_schema_version); select c.schema_version from t as c;

  × unexpected token at SourceSpan { offset: SourceOffset(47), length: 1 }
   ╭────
 1 │ with t as (select * from pragma_schema_version); select c.schema_version from t as c;
   ·                                                ┬
   ·                                                ╰── here
   ╰────
  help: expected [TK_SELECT, TK_VALUES, TK_UPDATE, TK_DELETE, TK_INSERT, TK_REPLACE] but found TK_SEMI
```
Now:
```text
 turso> with t as (select * from pragma_schema_version); select c.schema_version from t as c;

  × unexpected token ';' at offset 47
   ╭────
 1 │ with t as (select * from pragma_schema_version);select c.schema_version from t as c;
   ·                                                ┬
   ·                                                ╰── here
   ╰────
  help: expected SELECT, VALUES, UPDATE, DELETE, INSERT, or REPLACE but found ';'
  ```
@TcMits WDYT?

Closes #3190
2025-10-22 10:57:54 +03:00
Nikita Sivukhin
7d423d358f avoid unnecessary time measures 2025-10-22 11:08:46 +04:00
Pekka Enberg
d45a8da2f2 Merge 'parser: translate boolean values to literal when parsing column constraints' from Preston Thorpe
closes #3796

Closes #3804
2025-10-22 09:11:03 +03:00
Pekka Enberg
1aad1b224a Merge 'core/io: Make random generation deterministically simulated' from Pedro Muniz
Depends on #3584 to use the most up-to-date implementation of
`ThreadRng`
- Add `fill_bytes` method to `IO`
- use `thread_rng` instead of `getrandom`, as `getrandom` is much slower
and `thread_rng` offers enough security
- modify `exec_randomblob`, `exec_random` and random_rowid generation to
use methods from IO for determinism
- modified simulator IO to implement `fill_bytes`
This the PRNG for sqlite if someone is curious. It is similar to
`thread_rng`:
```c
/* Initialize the state of the random number generator once,
  ** the first time this routine is called.
  */
  if( wsdPrng.s[0]==0 ){
    sqlite3_vfs *pVfs = sqlite3_vfs_find(0);
    static const u32 chacha20_init[] = {
      0x61707865, 0x3320646e, 0x79622d32, 0x6b206574
    };
    memcpy(&wsdPrng.s[0], chacha20_init, 16);
    if( NEVER(pVfs==0) ){
      memset(&wsdPrng.s[4], 0, 44);
    }else{
      sqlite3OsRandomness(pVfs, 44, (char*)&wsdPrng.s[4]);
    }
    wsdPrng.s[15] = wsdPrng.s[12];
    wsdPrng.s[12] = 0;
    wsdPrng.n = 0;
  }

  assert( N>0 );
  while( 1 /* exit by break */ ){
    if( N<=wsdPrng.n ){
      memcpy(zBuf, &wsdPrng.out[wsdPrng.n-N], N);
      wsdPrng.n -= N;
      break;
    }
    if( wsdPrng.n>0 ){
      memcpy(zBuf, wsdPrng.out, wsdPrng.n);
      N -= wsdPrng.n;
      zBuf += wsdPrng.n;
    }
    wsdPrng.s[12]++;
    chacha_block((u32*)wsdPrng.out, wsdPrng.s);
    wsdPrng.n = 64;
  }
  sqlite3_mutex_leave(mutex);
```

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #3799
2025-10-22 09:10:36 +03:00
Pekka Enberg
9005458aa2 Merge 'antithesis-tests: Don't fail tests on ProgrammingError' from Pekka Enberg
The ProgrammingError exception is thrown when tables, indexes, or
columns are dropped in parallel. Let's not fail the Antithesis test
drivers when that happens.

Closes #3803
2025-10-22 09:10:30 +03:00
PThorpe92
2f401c0bcc Add regression tcl test for #3796 default bool col constraints 2025-10-21 21:22:09 -04:00
PThorpe92
892fcc881d Handle TRUE|FALSE literal case for default column constraint in the parser 2025-10-21 21:15:35 -04:00
Pere Diaz Bou
3227caaa1d Merge 'core: move BTreeCursor under MVCC cursor' from Pere Diaz Bou
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3756
2025-10-21 19:20:49 +02:00
Nikita Sivukhin
10ead9f3b6 one more clippy fix 2025-10-21 21:10:58 +04:00
pedrocarlo
72baf48863 add random generation in simulator IO 2025-10-21 14:10:38 -03:00
pedrocarlo
8c0b9c6979 add additional fill_bytes method to IO to deterministically generate
random bytes and modify random functions to use them
2025-10-21 14:10:38 -03:00
pedrocarlo
8501bc930a use workspace rand version 2025-10-21 14:10:05 -03:00
Pekka Enberg
4805c6b06b antithesis-tests: Don't fail tests on ProgrammingError
The ProgrammingError exception is thrown when tables, indexes, or
columns are dropped in parallel. Let's not fail the Antithesis test
drivers when that happens.
2025-10-21 20:08:11 +03:00