Commit Graph

10010 Commits

Author SHA1 Message Date
PThorpe92
16d19fd39e Add tcl tests for foreign keys 2025-10-07 16:28:04 -04:00
PThorpe92
2db18f8230 Add fk_fuzzing sql file to .gitignore 2025-10-07 16:28:04 -04:00
PThorpe92
346e6fedfa Create ForeignKey, ResolvedFkRef types and FK resolution 2025-10-07 16:27:49 -04:00
PThorpe92
c2b7026131 Add FOREIGN_KEY constraint error 2025-10-07 16:22:20 -04:00
PThorpe92
d04b07b8b7 Add pragma foreign_keys and fk_if_zero and fk_counter opcodes 2025-10-07 16:22:20 -04:00
PThorpe92
b40e784903 Update COMPAT.md, add fk related opcodes 2025-10-07 16:22:15 -04:00
Jussi Saurio
8bdef12e15 Merge 'Remove unsafe pointers (RawSlice) from RefValue' from Levy A.
- Add a lifetime parameter to `RefValue`, improving safety semantics by
preventing invalid pointers to `ImmutableRecord`.
- Also renames `RefValue` to `ValueRef`, to align with rusqlite and
other crates.
- Improve ergonomics by removing `RawSlice` in favor of native slices
Making it easier and safer to implement
https://github.com/tursodatabase/turso/issues/2304.
`TextSubtype` is stored as part of the enum variant of `ValueRef::Text`,
but this will be changed in a more general reworking of subtyping
described in https://github.com/tursodatabase/turso/issues/3573

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3587
2025-10-07 22:42:52 +03:00
Jussi Saurio
1cef12ef73 Merge 'Simulator: Add Drop and pave the way for Schema changes' from Pedro Muniz
Depends on #3585
Some properties can have extensional queries that run in between the
queries that the property aims to prove. These queries were generated
eagerly together with the generation of the `Property`. This was okayish
when we were not generating `Drop` queries, however with `Drop`
statements in the game we could generate queries that reference dropped
tables.
Example:
- Drop Table t;
- Select * from t;
The example above was possible because we update the simulator model
only after we run the query, so we could generate queries with stale
data.
**WHAT CHANGED**
- Stop generating queries eagerly in `Property`.
- Introduce `Query::Placeholder` to signify that the `Query` should be
generated in `PlanGenerator::next`. We then swap `Query::Placeholder`
with whatever query we generate
- This change is still compatible with MVCC as we still generate
`Commit` queries when `PlanGenerator` encounters a `DDL` statement
- Add `Property::AllTablesHaveExpectedContent` to check the tables in
the DB after a Faulty property executes, instead of pre selecting the
tables we want to check. We need to do this because a `FaultyQuery` may
Drop a table, resulting in a ParseError later on in the checks.
PS: In commit[`3c85040b4a483f4160d7324e664782a112a6a7a3`](https://github
.com/tursodatabase/turso/commit/3c85040b4a483f4160d7324e664782a112a6a7a3
), for correctness, I thought we had to clone the Simulator Tables every
time we wanted to generate extensional queries. However, later on I
reused the code of that commit and found a solution that could
generalize easier to any type of schema change. This will make it easier
for me add `ALTER TABLE` next.

Closes #3605
2025-10-07 22:41:53 +03:00
pedrocarlo
300d918040 fix differential check for parse error 2025-10-07 15:03:50 -03:00
pedrocarlo
3b2583c540 adjust Interaction generation to take into account possibilty of PropertyDistribution to have 0 Weights 2025-10-07 14:17:37 -03:00
pedrocarlo
c578f7ba96 Faultless should produce any type of query, just not faulty 2025-10-07 14:15:51 -03:00
pedrocarlo
21fc8bae2a Property::FaultyQuery and FsyncNoWait stored a list of tables to check the on the database. Again, the FaultyQuery could be a Drop Table which meant that we could be running a SELECT on an inexistent table. To solve this, just insert a Property that check all the tables in the db after a Faulty Property 2025-10-07 13:23:35 -03:00
pedrocarlo
6bad5d04ce generate extensional queries when iterating over the next interaction, not when generating the property. This is necessary as the extensional queries can modify schema and thus could cause the next queries to fail because the DB enviroment context was not updated on generation time. Rule of thumb: queries should never be generated in bulk, always one a a time so the enviroment can be shadowed accordingly 2025-10-07 13:19:53 -03:00
pedrocarlo
6d5443d4f0 add Query::Placeholder 2025-10-07 11:38:47 -03:00
pedrocarlo
7eb504baef certain properties cannot be generated if there are no tables in the current context 2025-10-07 11:38:47 -03:00
pedrocarlo
07cc1c548b adjust query generation to avoid DROP for certain extensional queries 2025-10-07 11:38:47 -03:00
pedrocarlo
4fc7be5042 as we have DROP table now, if we want to generate extensional queries eagerly, without affecting how we document interactions with MVCC, we need to travel forward in time and shadow queries eagerly so we can generate queries correctly. This involves cloning the tables unfortunately which is inneficient but correct 2025-10-07 11:38:47 -03:00
pedrocarlo
3e8867c8f5 DropSelect property should only fail when error is not a parse error on the table name 2025-10-07 11:38:47 -03:00
pedrocarlo
7f93f64fc5 enable Drop statements 2025-10-07 11:38:47 -03:00
Pekka Enberg
ff67c1f3ca Merge 'Make table name not repeat in simulator' from bit-aloo
closes: #3254

Closes #3617
2025-10-07 17:04:23 +03:00
Levy A.
cf53ecb7e3 refactor: remove TextRef and RawSlice and fix tests 2025-10-07 10:43:45 -03:00
Levy A.
77a412f6af refactor: remove unsafe reference semantics from RefValue
also renames `RefValue` to `ValueRef`, to align with rusqlite and other
crates
2025-10-07 10:43:44 -03:00
Pekka Enberg
243a5c719f Merge 'Fix re-entrancy of op_destroy (used by DROP TABLE)' from Jussi Saurio
op_destroy was assuming we never yield IO from
BTreeCursor::btree_destroy(), so every so often it would just not
complete the procedure and leave dangling pages in the database
Closes #3608

Closes #3618
2025-10-07 16:13:23 +03:00
bit-aloo
7f8c139638 make clippy happy 2025-10-07 18:27:02 +05:30
Jussi Saurio
5583e76981 Fix re-entrancy of op_destroy (used by DROP TABLE)
op_destroy was assuming we never yield IO from BTreeCursor::btree_destroy(),
so every so often it would just not complete the procedure and leave
dangling pages in the database
2025-10-07 15:38:30 +03:00
bit-aloo
73efe5d853 make table name not repeat in simulator 2025-10-07 17:49:15 +05:30
Pekka Enberg
fc2d532d80 Merge 'perf/throughput: Delete database before benchmark run' from Pekka Enberg
Let's reduce variance between benchmark runs by starting with an empty
database.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3610
2025-10-07 13:44:34 +03:00
Pekka Enberg
1dbf493f14 Merge 'emit proper column information for explain prepared statements' from Nikita Sivukhin
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3612
2025-10-07 13:44:17 +03:00
Pekka Enberg
b98c4ece0d Merge 'core/mvcc/logical-log: switch RwLock to parking_lot' from Pere Diaz Bou
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3615
2025-10-07 13:43:39 +03:00
Pekka Enberg
c3dbcec8c9 Merge 'core/io: remove new_dummy in place of new_yield' from Pere Diaz Bou
Yield is a completion that does not allocate any inner state. By design
it is completed from the start and has no errors. This allows lightly
yield without allocating any locks nor heap allocate inner state.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3616
2025-10-07 13:39:19 +03:00
Pere Diaz Bou
a7d2462c05 core/io/uring: fix inner usages
Yield is a completion that does not allocate any inner state. By design
it is completed from the start and has no errors. This allows lightly
yield without allocating any locks nor heap allocate inner state.
2025-10-07 12:05:54 +02:00
Pere Diaz Bou
3e508a4b42 core/io: remove new_dummy in place of new_yield
Yield is a completion that does not allocate any inner state. By design
it is completed from the start and has no errors. This allows lightly
yield without allocating any locks nor heap allocate inner state.
2025-10-07 12:00:33 +02:00
Pere Diaz Bou
44152f11d0 core/mvcc/logical-log: switch RwLock to parking_lot 2025-10-07 11:15:48 +02:00
Nikita Sivukhin
bd1013d62f emit proper column information for explain prepared statements 2025-10-07 12:28:55 +04:00
Pekka Enberg
d5c49c17c7 perf/throughput: Delete database before benchmark run
Let's reduce variance between benchmark runs by starting with an empty
database.
2025-10-07 09:57:08 +03:00
Jussi Saurio
f78103cafa Merge 'Modify Interactions Generation to only generate possible queries' from Pedro Muniz
I simplified `Interactions` generation code to first check the possible
queries we can generate and then create a Weighted Distribution to
generate `Query` and `Property`. This change allows us to not generate
certain types of queries if we have no tables, as it is not sound for
the Simulator to generate an `Update` query if there are no tables in
the database. I made this change for correctness and because I want to
introduce schema changes to the simulator and this is necessary.
Also, there is the additional benefit of us allocating less memory,
doing static function dispatch instead of creating closures all the time
for generation, and the code, in my opinion, being more readable.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Reviewed-by: bit-aloo (@Shourya742)

Closes #3585
2025-10-07 09:36:10 +03:00
Pekka Enberg
a72b07e949 Merge 'Fix VDBE program abort' from Nikita Sivukhin
This PR add proper program abort in case of unfinished statement reset
and interruption.
Also, this PR makes rollback methods non-failing because otherwise of
their callers usually unclear (if rollback failed - what is the state of
statement/connection/transaction?)

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3591
2025-10-07 09:07:07 +03:00
Pekka Enberg
dacb8e3350 Merge 'Fix attach I/O error with in-memory databases' from Preston Thorpe
closes #3540

Closes #3602
2025-10-07 09:00:02 +03:00
Pekka Enberg
ce41bebaa8 Merge 'core/incremental: Fix re-insertion of data with same key' from Glauber Costa
There is currently a bug found in our materialized view implementation
that happens when we delete a row, and then re-insert another row with
the same primary key.
Our insert code needs to detect updates and generate a DELETE + INSERT.
But in this case, after the initial DELETE, the fresh insert generates
another delete.
We ended up with the wrong response for aggregations (and I am pretty
sure even filter-only views would manifest the bug as well), where
groups that should still be present just disappeared because of the
extra delete.
A new test case is added that fails without the fix.

Closes #3601
2025-10-07 08:47:38 +03:00
Pekka Enberg
e9be38c769 Merge 'Add MVCC checkpoint threshold pragma' from bit-aloo
closes: #3575

Closes #3604
2025-10-07 08:46:41 +03:00
Pekka Enberg
5f2beb1122 Merge 'core/incremental: Implement "is null" and "is not null" tests for view filter' from Glauber Costa
Just overlook on our side that they were not generated before.

Closes #3603
2025-10-07 08:45:45 +03:00
pedrocarlo
8b6456f843 do not allow Property::Queries to attempt to be generated 2025-10-07 02:36:14 -03:00
pedrocarlo
91da12390d refactor property generation to use query distribution and avoid more
memory allocations
2025-10-07 02:36:14 -03:00
pedrocarlo
b1c26505b8 adjust Rng generic to include ?Sized + introduce WeightedDistribution trait 2025-10-07 02:36:13 -03:00
pedrocarlo
aec6668725 add ?Sized to Arbitrary traits 2025-10-07 02:36:13 -03:00
pedrocarlo
a5845285be remove unnecessary functions 2025-10-07 02:36:13 -03:00
pedrocarlo
bb9c8dea4f rework interaction generation to only generate possible queries + do less allocations 2025-10-07 02:36:13 -03:00
pedrocarlo
1d1b09dc17 modify query generation to always sample from valid queries 2025-10-07 02:36:13 -03:00
bit-aloo
afadb32c4c fmt fixes 2025-10-07 10:20:13 +05:30
bit-aloo
68b6ffe57c Implement mvcc_checkpoint_threshold pragma 2025-10-07 10:17:05 +05:30