Commit Graph

163 Commits

Author SHA1 Message Date
Pekka Enberg
8204fbc8ec simulator: Fix 64-bit offset build failures
Fix brekage from first merging commit d959319b ("Merge 'Use u64 for file
offsets in I/O and calculate such offsets in u64' from Preston Thorpe")
and then commit 6591b66c ("Merge 'Simulate I/O in memory' from Pedro
Muniz"), which was unaware of the changes.
2025-09-02 14:14:04 +03:00
Pekka Enberg
6591b66c3d Merge 'Simulate I/O in memory' from Pedro Muniz
Revives the `MemorySim` PR and fixes a page cache issue where we could
have a unlocked and unloaded page in the page cache after a FaultyQuery.
The page would continue in the cache and could affect other queries as
the `page_cache` is at the `Connection` level.
Depends on #2785

Closes #2693
2025-09-02 13:28:48 +03:00
Pekka Enberg
d959319b42 Merge 'Use u64 for file offsets in I/O and calculate such offsets in u64' from Preston Thorpe
Using `usize` to compute file offsets caps us at ~16GB on 32-bit
systems. For example, with 4 KiB pages we can only address up to 1048576
pages; attempting the next page overflows a 32-bit usize and can wrap
the write offset, corrupting data. Switching our I/O APIs and offset
math to u64 avoids this overflow on 32-bit targets

Closes #2791
2025-09-02 09:06:49 +03:00
pedrocarlo
51a54d3c33 Fd should be part of Operation struct 2025-09-01 16:53:07 -03:00
pedrocarlo
1eb1171f55 do not fault on Fsync until we correctly define the expected behaviour in the simulator 2025-09-01 14:12:11 -03:00
pedrocarlo
c158db072b inject fault in the IO Operation in the MemorySim 2025-09-01 14:12:11 -03:00
pedrocarlo
bd8cfe40f1 impl remove_file for MemoryIO + skip completion if finished 2025-09-01 14:12:11 -03:00
pedrocarlo
cc3488bba0 print path in stats for memory IO 2025-09-01 14:12:11 -03:00
pedrocarlo
fd4e74929a Cli option to enable memory IO 2025-09-01 14:12:11 -03:00
pedrocarlo
117451fba1 impl WriteV for MemorySim 2025-09-01 14:12:11 -03:00
pedrocarlo
24c0d24be6 impl Truncate for MemorySim 2025-09-01 11:11:25 -03:00
pedrocarlo
40de4c0606 initial impl for MemorySim 2025-09-01 11:11:25 -03:00
pedrocarlo
8c7da3a704 impl SimIO for SimulatorIO 2025-09-01 11:11:03 -03:00
pedrocarlo
d8e9f145e6 create SimIO trait 2025-09-01 11:10:40 -03:00
pedrocarlo
5881ee71d6 clippy 2025-08-30 12:21:37 -03:00
pedrocarlo
961c0cd282 script to save JsonSchema for editor integration 2025-08-30 12:17:50 -03:00
pedrocarlo
61fa7546c1 fold some SimulatorOpts fields to Profile 2025-08-30 11:31:52 -03:00
pedrocarlo
2f237fdcfd adjust remaining calculation to use the profile 2025-08-30 11:31:52 -03:00
pedrocarlo
962666831b read Profile file from path or use predefined profiles 2025-08-30 11:31:52 -03:00
pedrocarlo
e0552629e3 create Generation Options structs 2025-08-30 11:31:52 -03:00
pedrocarlo
bcd70488ae add sqlite integrity check back 2025-08-29 12:25:27 -03:00
PThorpe92
0a56d23402 Use u64 for file offsets in IO and calculate such offsets in u64 2025-08-28 09:44:00 -04:00
pedrocarlo
8010b7d0c7 make simulator use sql_generation crate as dependency 2025-08-25 22:59:31 -03:00
Jussi Saurio
cc643362a4 sim: remove "run_once faults"
This kind of fault does not semantically represent anything real,
since we already have fault injection for every concrete IO operation
like reading, writing, syncing and so forth.

Moreover, having this feature is the direct cause of the false positive
simulator failure as reported in issue #2727. There, a "run_once fault"
happened immediately after we fsynced following an INSERT, which caused
the simulator to think the INSERT failed, and later a sim assertion failed
because the on-disk database had 1 more row than it thought it would.
2025-08-22 10:13:06 +03:00
Nikita Sivukhin
c771487933 add remove_file method to the IO 2025-08-21 14:51:02 +04:00
Jussi Saurio
e6adb8992b sim: use 'git rev-parse --show-toplevel' for getting base dir 2025-08-20 09:58:21 +03:00
pedrocarlo
7bc0545442 default impl for get_memory_io 2025-08-19 10:48:21 -03:00
pedrocarlo
f72bcbc5da default impl for wait_for_completion + check for errors in completion there 2025-08-19 10:48:21 -03:00
pedrocarlo
d96a26aef9 Property TableHasExpectedContent should just check the expected
content on runtime, not generation time
2025-08-18 16:00:59 -03:00
pedrocarlo
7fb14cfc76 add File path to SimulatorFile 2025-08-18 11:52:10 -03:00
pedrocarlo
6388ed2017 FaultyQuery enabled by default 2025-08-18 11:52:10 -03:00
PThorpe92
2810b5d8a2 Add --keep-files flag and force Drop of database in simulator to prevent issues with the Registry 2025-08-15 17:54:40 -04:00
pedrocarlo
0c9ecbc768 simulator should delete files after a successful run 2025-08-09 16:25:51 -03:00
PThorpe92
f6a68cffc2 Remove RefCell from IO and Page apis 2025-08-05 16:24:49 -04:00
pedrocarlo
543cdb3e2c underscoring completions and IOResult to avoid warning messages 2025-07-31 11:51:17 -03:00
PThorpe92
daec8aeb22 impl pwritev for simulator file 2025-07-30 19:42:49 -04:00
PThorpe92
9c75872827 Make writing wal header able to be sync 2025-07-30 14:05:18 +03:00
PThorpe92
dcf346b9b0 Update apparently very old branch to new api 2025-07-30 13:48:05 +03:00
PThorpe92
52f63b2af0 Impl truncate for simulator 2025-07-30 13:48:05 +03:00
pedrocarlo
3831e0db39 convert must_use compile warnings to unused_variables to track locations where we need to refactor in the future 2025-07-28 16:09:26 -03:00
pedrocarlo
d30c7d54c8 change all Arc<Completion> to Completion 2025-07-28 15:32:45 -03:00
Jussi Saurio
e604a80e93 fix/sim: actually enable indexes by default 2025-07-21 13:35:11 +03:00
Jussi Saurio
dce355d9b7 test/stress&sim: enable indexes by default 2025-07-21 11:49:30 +03:00
Jussi Saurio
b1ebc1c82f Merge 'sim: show seed in 'execute_interaction()' trace' from Jussi Saurio
This helps e.g. when you hit some infinite loop

Closes #2161
2025-07-18 12:27:48 +03:00
Jussi Saurio
687d126e25 sim: change --disable-create-index flag to --enable-create-index (default false) 2025-07-18 11:21:24 +03:00
Jussi Saurio
5b2d7315eb sim: show seed in 'execute_interaction()' trace 2025-07-18 11:21:04 +03:00
Jussi Saurio
1b52b5c764 Merge 'chore: update rust to version 1.88.0' from Nils Koch
This PR updates to version Rust 1.88.0 ([Release
notes](https://releases.rs/docs/1.88.0/)) and fixes all the clippy
errors that come with the new Rust version.
This is possible in the latest Rust version:
```rust
if let Some(foo) = bar && foo.is_cool() {
  ...
}
```
There are three complications in the migration (so far):
- A BUNCH of Clippy warnings (mostly fixed in
https://github.com/tursodatabase/limbo/pull/1827)
- Windows cross compilation failed; linking `advapi32` on windows fixes
it
  - Since Rust 1.87.0, advapi32 is not linked by default anymore
([Release notes](https://github.com/rust-
lang/rust/blob/master/RELEASES.md#compatibility-notes-1),
[PR](https://github.com/rust-lang/rust/pull/138233))
- Rust is more strict with FFIs and aligning pointers now. CI checks
failed with error below
  - Fixed in https://github.com/tursodatabase/turso/pull/2064
```
thread 'main' panicked at
core/ext/vtab_xconnect.rs:64:25:
misaligned pointer dereference: address must be
a multiple of 0x8 but is 0x7ffd9d901554
```

Closes #1807
2025-07-17 23:35:33 +03:00
pedrocarlo
b80218324d fix merge conflicts 2025-07-17 12:25:31 -03:00
pedrocarlo
6088aa34c2 only add the sync completion after we call sync in the closure 2025-07-17 12:25:31 -03:00
pedrocarlo
0ab2f2b951 cli options for max and min tick + adjust github action to run with faster clock so no timeouts happen 2025-07-17 12:25:31 -03:00