If we don't clear the dirty pages, we will initiate a rollback. In the
rollback, we will attempt to clear the whole page cache, but it will
then panic because there will still be dirty pages from the failed
writev
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#3189
Nyrkiö is piloting a GitHub Runner service, and congratulations, you are
the pilot customer! In order to reduce the risk somewhat, we'll
introduce this as a parallel workflow, so the existing benchmarks will
continue to run on the regular runners and they won't have any
discontinuity in their results because of this. Also since these runners
are actually a bit more expensive, we can manage the cost by only
running them periodically. Similarly, this allows us to fine tune the
Nyrkiö instance size, for example to have an EBS disk that is the
optimal size and IOPS configuration, so you don't pay for empty space.
I will show up in discord to discuss.
And congratulations on the beta release by the way! I worked hard to get
this done for your beta period, let's hope it works now.
Closes#3619
Rollback shouldn't modify the row version chain. This is crucial for
implementing a Non-blocking row version chain in #3499
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#3583
SQLite surprisingly supports this:
select sqlite_version(*);
this gets translated at the parser level to sqlite_version(), and it
works for all functions that take 0 arguments.
Let's be compatible with SQLite and support the same thing.
Closes#3630
This PR introduces support for foreign key constraints, and the `PRAGMA
foreign_keys;`, and relevant opcodes: `FkCounter` and `FkIfZero`.
Extensive fuzz tests were added both for regular and composite
PK/rowid/unique index constraints, as well as some really weird
edgecases to make sure we our affinity handling is correct as well when
we trigger the constraints.
Foreign-key checking is driven by two VDBE ops: `FkCounter` and
`FkIfZero`, and
`FkCounter` is a running meter on the `Connection` for deferred FK
violations. When an `insert/delete/update` operation creates a potential
orphan (we insert a child row that doesn’t have a matching parent, or we
delete/update a parent that children still point at), this counter is
incremented. When a later operation fixes that (e.g. we insert the
missing parent or re-target the child), we decrement the counter. If any
is remaining at commit time, the commit fails. For immediate
constraints, on the violation path we emit Halt right away.
`FkIfZero` can either be used to guard a decrement of FkCounter to
prevent underflow, or can potentially (in the future) be used to avoid
work checking if any constraints need resolving.
NOTE: this PR does not implement `pragma defer_foreign_keys` for global
`deferred` constraint semantics. only explicit `col INT REFERENCES t(id)
DEFERRABLE INITIALLY DEFERRED` is supported in this PR.
This PR does not add support for `ON UPDATE|DELETE CASCADE`, only for
basic implicit `DO NOTHING` behavior.
~~NOTE: I did notice that, as referenced here: #3463~~
~~our current handling of unique constraints does not pass fuzz tests, I
believe only in the case of composite primary keys,~~ ~~because the fuzz
test for FK referencing composite PK is failing but only for UNIQUE
constraints, never (or as many times as i tried) for foreign key
constraints.~~
EDIT: all fuzzers are passing, because @sivukhin fixed the unique
constraint issue.
The reason that the `deferred` fuzzer is `#[ignore]`'d is because sqlite
uses sub-transactions, and even though the fuzzing only does 1 entry per
transaction... the fuzzer can lose track of _when_ it's in a transaction
and when it hits a FK constraint, and there is an error in both DB's, it
can just continue to do run regular statements, and then the eventual
ROLLBACK will revert different things in sqlite vs turso.. so for now,
we leave it `ignore`d
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#3510
SQLite surprisingly supports this:
select sqlite_version(*);
this gets translated at the parser level to sqlite_version(), and it
works for all functions that take 0 arguments.
Let's be compatible with SQLite and support the same thing.
- add index root pages to list of root pages to check
- check for dangling (unused) pages
```sql
$ cargo run wut.db
turso> .mode list
turso> pragma integrity_check;
Page 3: never used
Page 4: never used
Page 7: never used
Page 8: never used
```
```sql
$ sqlite3 wut.db 'pragma integrity_check;'
*** in database main ***
Page 3: never used
Page 4: never used
Page 7: never used
Page 8: never used
```
Closes#3613
- Add a lifetime parameter to `RefValue`, improving safety semantics by
preventing invalid pointers to `ImmutableRecord`.
- Also renames `RefValue` to `ValueRef`, to align with rusqlite and
other crates.
- Improve ergonomics by removing `RawSlice` in favor of native slices
Making it easier and safer to implement
https://github.com/tursodatabase/turso/issues/2304.
`TextSubtype` is stored as part of the enum variant of `ValueRef::Text`,
but this will be changed in a more general reworking of subtyping
described in https://github.com/tursodatabase/turso/issues/3573
Reviewed-by: Preston Thorpe <preston@turso.tech>
Closes#3587
Depends on #3585
Some properties can have extensional queries that run in between the
queries that the property aims to prove. These queries were generated
eagerly together with the generation of the `Property`. This was okayish
when we were not generating `Drop` queries, however with `Drop`
statements in the game we could generate queries that reference dropped
tables.
Example:
- Drop Table t;
- Select * from t;
The example above was possible because we update the simulator model
only after we run the query, so we could generate queries with stale
data.
**WHAT CHANGED**
- Stop generating queries eagerly in `Property`.
- Introduce `Query::Placeholder` to signify that the `Query` should be
generated in `PlanGenerator::next`. We then swap `Query::Placeholder`
with whatever query we generate
- This change is still compatible with MVCC as we still generate
`Commit` queries when `PlanGenerator` encounters a `DDL` statement
- Add `Property::AllTablesHaveExpectedContent` to check the tables in
the DB after a Faulty property executes, instead of pre selecting the
tables we want to check. We need to do this because a `FaultyQuery` may
Drop a table, resulting in a ParseError later on in the checks.
PS: In commit[`3c85040b4a483f4160d7324e664782a112a6a7a3`](https://github
.com/tursodatabase/turso/commit/3c85040b4a483f4160d7324e664782a112a6a7a3
), for correctness, I thought we had to clone the Simulator Tables every
time we wanted to generate extensional queries. However, later on I
reused the code of that commit and found a solution that could
generalize easier to any type of schema change. This will make it easier
for me add `ALTER TABLE` next.
Closes#3605
