Commit Graph

104 Commits

Author SHA1 Message Date
Pekka Enberg
99d61aad3c simulator: Add mention of fsync() issue for disabled fsync faults 2025-07-16 11:15:41 +03:00
Pekka Enberg
b0971f98c2 Merge 'sim: ignore fsync faults' from Jussi Saurio
`FaultyQuery` causes frequent false positives in simulator due to the
following chain of events:
- we write rows and flush wal to disk
- inject fault during fsync which fails
- error is returned to caller, simulator thinks those rows dont exist
because the query failed
- we reopen the database i.e. read the WAL back to memory from disk, it
has those extra rows we think we didn't write
- assertion fails because table has more rows than simulator expected
More discussion about fsync behavior in issue #2091

Closes #2110
2025-07-16 11:15:23 +03:00
Jussi Saurio
bb0cad459e sim: ignore fsync faults
`FaultyQuery` causes frequent false positives in simulator due to
the following chain of events:

- we write rows and flush wal to disk
- inject fault during fsync which fails
- error is returned to caller, simulator thinks those rows dont exist because the query failed
- we reopen the database i.e. read the WAL back to memory from disk, it has those extra rows we think we didn't write
- assertion fails because table has more rows than simulator expected

More discussion about fsync behavior in issue #2091
2025-07-16 11:09:54 +03:00
Pekka Enberg
1a8bade9d5 Merge 'Updates to the simulator' from Alperen Keleş
- Add generation for UNION/JOIN
- Rearchitect the oracle calling conventions to simplify the code paths
- Add brute force shrinking option by @echoumcp1

Closes #2049
2025-07-16 11:03:41 +03:00
alpaylan
6b96789b6d add random_expr for SELECT <expr>; 2025-07-14 18:48:02 -04:00
Pere Diaz Bou
88ff218810 io: assert small I/O
Let's assert **for now** that we do not read/write less bytes than
expected. This should be fixed to retrigger several reads/writes if we
couldn't read/write enough but for now let's assert.
2025-07-14 10:19:41 +02:00
alpaylan
0b3fb2ecdd Merge branch 'main' of https://github.com/tursodatabase/limbo 2025-07-13 11:10:52 -04:00
Nils Koch
828d4f5016 fix clippy errors for rust 1.88.0 (auto fix) 2025-07-12 18:58:41 +03:00
Jussi Saurio
cb16301157 sim: return LimboError::Busy when busy, instead of looping forever 2025-07-12 10:24:49 +03:00
alpaylan
e28ad97ac7 cargo fmt and cargo clippy fixes 2025-07-11 03:35:32 -04:00
alpaylan
3a967f9127 remove rusqlite integrity check, sync generation with --experimental-indexes, move FaultyQuery to disabled by default 2025-07-11 03:35:02 -04:00
alpaylan
b98542d112 cargo fmt and cargo clippy fixes 2025-07-11 02:12:56 -04:00
alpaylan
65fe60ba57 fix the merge conflicts 2025-07-11 02:04:14 -04:00
alpaylan
8f46bbc77e Merge branch 'main' of https://github.com/tursodatabase/limbo 2025-07-11 01:33:50 -04:00
alpaylan
489602b095 updates to the oracle implementations for differential and doublecheck, add integrity check to the end of the simulation, run cargo clippy --fix 2025-07-11 01:22:40 -04:00
alpaylan
a7b5a2c435 Merge branch 'main' of https://github.com/alpaylan/limbo 2025-07-09 00:14:24 -04:00
alpaylan
44a2fa60a3 add union all cardinality preservation, start generating INSERT INTO t SELECT... queries, add integrity checks to the end of run_simulation 2025-07-09 00:14:13 -04:00
echoumcp1
c5d719dafa adding better shrinking and cli opts 2025-07-07 21:21:21 -04:00
pedrocarlo
d21a629cd9 rollback simulator table when we encounter a Rollback query 2025-07-07 13:37:51 -03:00
pedrocarlo
7c8737e292 do not shadow + continue the assertion on injected fault error 2025-07-07 11:53:45 -03:00
pedrocarlo
711b1ef114 make all run_once be run under statement or connection so that rollback is called 2025-07-07 11:51:25 -03:00
pedrocarlo
897426a662 add error tracing to relevant functions + rollback transaction in step_end_write_txn + make move_to_root return result 2025-07-07 11:50:21 -03:00
pedrocarlo
5f9abb62c4 enable faulty query 2025-07-07 11:50:21 -03:00
Pekka Enberg
931a33642e Merge 'add interactive transaction to property insert-values-select' from Pere Diaz Bou
Closes #1958
2025-07-07 14:29:02 +03:00
Jussi Saurio
ec939eaaa9 sim: add feature flags (indexes,mvcc) to CLI args 2025-07-07 10:27:15 +03:00
alpaylan
ea9c67a950 generate joins and unions 2025-07-07 02:54:48 -04:00
alpaylan
2124cc41c0 enable indexes 2025-07-06 14:51:20 -04:00
alpaylan
0bce68b38d wip: add joins to the select 2025-07-06 14:46:38 -04:00
Pere Diaz Bou
897f13c173 add interactive transaction to property insert-values-select 2025-07-04 15:50:46 +02:00
pedrocarlo
251926cd79 leave io faults and fsync query disabled by default 2025-06-30 12:36:26 -03:00
pedrocarlo
4a32f83bd0 add cli option to disable faulty query 2025-06-29 15:56:03 -03:00
pedrocarlo
119c537334 add FaultyQuery Property to inject fault in a query and then assert that it did not modify the database 2025-06-29 15:54:27 -03:00
pedrocarlo
b578f2249a add FaultyQuery 2025-06-29 14:22:11 -03:00
pedrocarlo
ac34928d53 fix delete shadowing + build result rows from fsync query 2025-06-29 12:44:13 -03:00
pedrocarlo
2461b4eb46 fix shadowing and run a query after the fsync query to try to catch a problem with the WAL 2025-06-29 12:40:53 -03:00
pedrocarlo
684a23020d add cli options to disable fsync property 2025-06-29 12:40:53 -03:00
pedrocarlo
0288f4aac6 introduce fsync interaction + property 2025-06-29 12:40:52 -03:00
pedrocarlo
f806d97d0f keep track when fsync is called 2025-06-29 12:40:23 -03:00
Pekka Enberg
53ba3ff926 Rename limbo_core crate to turso_core 2025-06-29 09:59:17 +03:00
pedrocarlo
9aa733f80c sleep inside Io completion 2025-06-26 22:17:28 -03:00
pedrocarlo
e2aafacbb4 add cli option to adjust latency probability 2025-06-26 22:17:27 -03:00
pedrocarlo
c8937976e5 generate_latency for each io operation on the file 2025-06-26 22:17:27 -03:00
pedrocarlo
56547f7127 change refcell values to cell 2025-06-26 22:17:27 -03:00
Pekka Enberg
2fc5c0ce5c Switch to runtime flag for enabling indexes
Makes it easier to test the feature:

```
$ cargo run --  --experimental-indexes
Limbo v0.0.22
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database
limbo> CREATE TABLE t(x);
limbo> CREATE INDEX t_idx ON t(x);
limbo> DROP INDEX t_idx;
```
2025-06-26 10:07:28 +03:00
Jussi Saurio
9aae1ff00a sim: add Fault::ReopenDatabase
Add fault type that reopens the DB and reconnects the connections.
It purposefully does not call conn.close(), as the default behavior
of that method is to checkpoint the database.

This easily exposes multiple manifestations of DB/WAL corruption caused by
this issue: https://github.com/tursodatabase/limbo/issues/1725

in fact, in my testing, every run fails when this fault is enabled. Hence I've
added the --disable-reopen-database flag if you want to try to find some other
bugs.
2025-06-24 14:41:50 -03:00
Jussi Saurio
73db450b8b sim: when loading bug, dont panic if there are no runs 2025-06-24 15:46:15 +03:00
Nils Koch
2827b86917 chore: fix clippy warnings 2025-06-23 19:52:13 +01:00
pedrocarlo
c9b11dd173 simulator integrity check per query 2025-06-22 21:47:17 -03:00
Pekka Enberg
90c1e3fc06 Switch Connection to use Arc instead of Rc
Connection needs to be Arc so that bindings can wrap it with `Mutex` for
multi-threading.
2025-06-16 10:43:19 +03:00
pedrocarlo
83d72db901 better error messages + cleaner printing of file stats + tracing
instrumentation in `execute_interaction`
2025-06-11 11:32:45 -03:00