Commit Graph

377 Commits

Author SHA1 Message Date
Pekka Enberg
1de647758f Merge 'refactor parser fmt' from Lâm Hoàng Phúc
@penberg this PR try to clean up `turso_parser`'s`fmt` code.
- `get_table_name` and `get_column_name` should return None when
table/column does not exist.
```rust
/// Context to be used in ToSqlString
pub trait ToSqlContext {
    /// Given an id, get the table name
    /// First Option indicates whether the table exists
    ///
    /// Currently not considering aliases
    fn get_table_name(&self, _id: TableInternalId) -> Option<&str> {
        None
    }

    /// Given a table id and a column index, get the column name
    /// First Option indicates whether the column exists
    /// Second Option indicates whether the column has a name
    fn get_column_name(&self, _table_id: TableInternalId, _col_idx: usize) -> Option<Option<&str>> {
        None
    }

    // help function to handle missing table/column names
    fn get_table_and_column_names(
        &self,
        table_id: TableInternalId,
        col_idx: usize,
    ) -> (String, String) {
        let table_name = self
            .get_table_name(table_id)
            .map(|s| s.to_owned())
            .unwrap_or_else(|| format!("t{}", table_id.0));

        let column_name = self
            .get_column_name(table_id, col_idx)
            .map(|opt| {
                opt.map(|s| s.to_owned())
                    .unwrap_or_else(|| format!("c{col_idx}"))
            })
            .unwrap_or_else(|| format!("c{col_idx}"));

        (table_name, column_name)
    }
}
```
- remove `FmtTokenStream` because it is same as `WriteTokenStream `
- remove useless functions and simplify `ToTokens`
```rust
/// Generate token(s) from AST node
/// Also implements Display to make sure devs won't forget Display
pub trait ToTokens: Display {
    /// Send token(s) to the specified stream with context
    fn to_tokens<S: TokenStream + ?Sized, C: ToSqlContext>(
        &self,
        s: &mut S,
        context: &C,
    ) -> Result<(), S::Error>;

    // Return displayer representation with context
    fn displayer<'a, 'b, C: ToSqlContext>(&'b self, ctx: &'a C) -> SqlDisplayer<'a, 'b, C, Self>
    where
        Self: Sized,
    {
        SqlDisplayer::new(ctx, self)
    }
}
```

Closes #2748
2025-09-02 18:35:43 +03:00
Pekka Enberg
8204fbc8ec simulator: Fix 64-bit offset build failures
Fix brekage from first merging commit d959319b ("Merge 'Use u64 for file
offsets in I/O and calculate such offsets in u64' from Preston Thorpe")
and then commit 6591b66c ("Merge 'Simulate I/O in memory' from Pedro
Muniz"), which was unaware of the changes.
2025-09-02 14:14:04 +03:00
TcMits
33a04fbaf7 resolve conflict 2025-09-02 17:30:10 +07:00
Pekka Enberg
6591b66c3d Merge 'Simulate I/O in memory' from Pedro Muniz
Revives the `MemorySim` PR and fixes a page cache issue where we could
have a unlocked and unloaded page in the page cache after a FaultyQuery.
The page would continue in the cache and could affect other queries as
the `page_cache` is at the `Connection` level.
Depends on #2785

Closes #2693
2025-09-02 13:28:48 +03:00
Pekka Enberg
d959319b42 Merge 'Use u64 for file offsets in I/O and calculate such offsets in u64' from Preston Thorpe
Using `usize` to compute file offsets caps us at ~16GB on 32-bit
systems. For example, with 4 KiB pages we can only address up to 1048576
pages; attempting the next page overflows a 32-bit usize and can wrap
the write offset, corrupting data. Switching our I/O APIs and offset
math to u64 avoids this overflow on 32-bit targets

Closes #2791
2025-09-02 09:06:49 +03:00
pedrocarlo
51a54d3c33 Fd should be part of Operation struct 2025-09-01 16:53:07 -03:00
pedrocarlo
1eb1171f55 do not fault on Fsync until we correctly define the expected behaviour in the simulator 2025-09-01 14:12:11 -03:00
pedrocarlo
c158db072b inject fault in the IO Operation in the MemorySim 2025-09-01 14:12:11 -03:00
pedrocarlo
bd8cfe40f1 impl remove_file for MemoryIO + skip completion if finished 2025-09-01 14:12:11 -03:00
pedrocarlo
cc3488bba0 print path in stats for memory IO 2025-09-01 14:12:11 -03:00
pedrocarlo
ff51965c3e fix shrinking to not include some properties that are present in faulty query, but that are not relevant to the error 2025-09-01 14:12:11 -03:00
pedrocarlo
fd4e74929a Cli option to enable memory IO 2025-09-01 14:12:11 -03:00
pedrocarlo
117451fba1 impl WriteV for MemorySim 2025-09-01 14:12:11 -03:00
pedrocarlo
24c0d24be6 impl Truncate for MemorySim 2025-09-01 11:11:25 -03:00
pedrocarlo
40de4c0606 initial impl for MemorySim 2025-09-01 11:11:25 -03:00
pedrocarlo
c01449e71b add parking_lot to simulator 2025-09-01 11:11:25 -03:00
pedrocarlo
8c7da3a704 impl SimIO for SimulatorIO 2025-09-01 11:11:03 -03:00
pedrocarlo
d8e9f145e6 create SimIO trait 2025-09-01 11:10:40 -03:00
pedrocarlo
4f2bc96dbe add Faultless profile 2025-08-30 13:07:19 -03:00
pedrocarlo
5881ee71d6 clippy 2025-08-30 12:21:37 -03:00
pedrocarlo
961c0cd282 script to save JsonSchema for editor integration 2025-08-30 12:17:50 -03:00
pedrocarlo
9aac45c3de small docs for profile 2025-08-30 11:31:52 -03:00
pedrocarlo
b9cc556a55 adjust write heavy profile to insert more rows 2025-08-30 11:31:52 -03:00
pedrocarlo
61fa7546c1 fold some SimulatorOpts fields to Profile 2025-08-30 11:31:52 -03:00
pedrocarlo
463eb1fefd simplify profile weights for writes 2025-08-30 11:31:52 -03:00
pedrocarlo
2f237fdcfd adjust remaining calculation to use the profile 2025-08-30 11:31:52 -03:00
pedrocarlo
962666831b read Profile file from path or use predefined profiles 2025-08-30 11:31:52 -03:00
pedrocarlo
a1407869d4 add serde, schemars and garde to profiles and options 2025-08-30 11:31:52 -03:00
pedrocarlo
06b923d0c1 adjust simulator to use correct trait signature 2025-08-30 11:31:52 -03:00
pedrocarlo
e0552629e3 create Generation Options structs 2025-08-30 11:31:52 -03:00
pedrocarlo
ef16bc4cfb add profiles together 2025-08-30 11:31:52 -03:00
pedrocarlo
918c2a3f69 extend latency profile + impl Default manually 2025-08-30 11:31:52 -03:00
pedrocarlo
19d9003cd7 create profiles folder 2025-08-30 11:31:52 -03:00
pedrocarlo
bcd70488ae add sqlite integrity check back 2025-08-29 12:25:27 -03:00
PThorpe92
0a56d23402 Use u64 for file offsets in IO and calculate such offsets in u64 2025-08-28 09:44:00 -04:00
TcMits
4ddfdb2a62 finish 2025-08-27 14:58:35 +07:00
pedrocarlo
8010b7d0c7 make simulator use sql_generation crate as dependency 2025-08-25 22:59:31 -03:00
Jussi Saurio
cc643362a4 sim: remove "run_once faults"
This kind of fault does not semantically represent anything real,
since we already have fault injection for every concrete IO operation
like reading, writing, syncing and so forth.

Moreover, having this feature is the direct cause of the false positive
simulator failure as reported in issue #2727. There, a "run_once fault"
happened immediately after we fsynced following an INSERT, which caused
the simulator to think the INSERT failed, and later a sim assertion failed
because the on-disk database had 1 more row than it thought it would.
2025-08-22 10:13:06 +03:00
Nikita Sivukhin
c771487933 add remove_file method to the IO 2025-08-21 14:51:02 +04:00
Jussi Saurio
e6adb8992b sim: use 'git rev-parse --show-toplevel' for getting base dir 2025-08-20 09:58:21 +03:00
pedrocarlo
7bc0545442 default impl for get_memory_io 2025-08-19 10:48:21 -03:00
pedrocarlo
f72bcbc5da default impl for wait_for_completion + check for errors in completion there 2025-08-19 10:48:21 -03:00
Jussi Saurio
97657a86b3 Do not assume error message content in FaultyQuery 2025-08-19 12:49:01 +03:00
pedrocarlo
d96a26aef9 Property TableHasExpectedContent should just check the expected
content on runtime, not generation time
2025-08-18 16:00:59 -03:00
pedrocarlo
2954e2e7bf shrinking: remove table assertions for non-dependent tables 2025-08-18 12:44:49 -03:00
pedrocarlo
59da828362 do not shadow FaultyQuery's immediately. Only shadow them later 2025-08-18 12:40:02 -03:00
pedrocarlo
7fb14cfc76 add File path to SimulatorFile 2025-08-18 11:52:10 -03:00
pedrocarlo
6388ed2017 FaultyQuery enabled by default 2025-08-18 11:52:10 -03:00
Jussi Saurio
6c17fa2a5e fix/sim: prevent sim from trying to create an existing table or index
We recently merged a change that panics the sim on parse errors, because
not doing so has masked many scenarios where the sim unintentionally
creates incorrect sql and we just ignore it.

We already have Property::DoubleCreateFailure to assert that the same table
cannot be created twice, so this should not hide any bugs.
2025-08-17 18:13:05 +03:00
Jussi Saurio
0a3ebf8914 Fail simulator on parse errors 2025-08-16 10:27:10 +03:00