This PR introduces support for foreign key constraints, and the `PRAGMA
foreign_keys;`, and relevant opcodes: `FkCounter` and `FkIfZero`.
Extensive fuzz tests were added both for regular and composite
PK/rowid/unique index constraints, as well as some really weird
edgecases to make sure we our affinity handling is correct as well when
we trigger the constraints.
Foreign-key checking is driven by two VDBE ops: `FkCounter` and
`FkIfZero`, and
`FkCounter` is a running meter on the `Connection` for deferred FK
violations. When an `insert/delete/update` operation creates a potential
orphan (we insert a child row that doesn’t have a matching parent, or we
delete/update a parent that children still point at), this counter is
incremented. When a later operation fixes that (e.g. we insert the
missing parent or re-target the child), we decrement the counter. If any
is remaining at commit time, the commit fails. For immediate
constraints, on the violation path we emit Halt right away.
`FkIfZero` can either be used to guard a decrement of FkCounter to
prevent underflow, or can potentially (in the future) be used to avoid
work checking if any constraints need resolving.
NOTE: this PR does not implement `pragma defer_foreign_keys` for global
`deferred` constraint semantics. only explicit `col INT REFERENCES t(id)
DEFERRABLE INITIALLY DEFERRED` is supported in this PR.
This PR does not add support for `ON UPDATE|DELETE CASCADE`, only for
basic implicit `DO NOTHING` behavior.
~~NOTE: I did notice that, as referenced here: #3463~~
~~our current handling of unique constraints does not pass fuzz tests, I
believe only in the case of composite primary keys,~~ ~~because the fuzz
test for FK referencing composite PK is failing but only for UNIQUE
constraints, never (or as many times as i tried) for foreign key
constraints.~~
EDIT: all fuzzers are passing, because @sivukhin fixed the unique
constraint issue.
The reason that the `deferred` fuzzer is `#[ignore]`'d is because sqlite
uses sub-transactions, and even though the fuzzing only does 1 entry per
transaction... the fuzzer can lose track of _when_ it's in a transaction
and when it hits a FK constraint, and there is an error in both DB's, it
can just continue to do run regular statements, and then the eventual
ROLLBACK will revert different things in sqlite vs turso.. so for now,
we leave it `ignore`d
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#3510
- add index root pages to list of root pages to check
- check for dangling (unused) pages
```sql
$ cargo run wut.db
turso> .mode list
turso> pragma integrity_check;
Page 3: never used
Page 4: never used
Page 7: never used
Page 8: never used
```
```sql
$ sqlite3 wut.db 'pragma integrity_check;'
*** in database main ***
Page 3: never used
Page 4: never used
Page 7: never used
Page 8: never used
```
Closes#3613
op_destroy was assuming we never yield IO from
BTreeCursor::btree_destroy(), so every so often it would just not
complete the procedure and leave dangling pages in the database
Closes#3608Closes#3618
op_destroy was assuming we never yield IO from BTreeCursor::btree_destroy(),
so every so often it would just not complete the procedure and leave
dangling pages in the database
Yield is a completion that does not allocate any inner state. By design
it is completed from the start and has no errors. This allows lightly
yield without allocating any locks nor heap allocate inner state.
Yield is a completion that does not allocate any inner state. By design
it is completed from the start and has no errors. This allows lightly
yield without allocating any locks nor heap allocate inner state.
This PR add proper program abort in case of unfinished statement reset
and interruption.
Also, this PR makes rollback methods non-failing because otherwise of
their callers usually unclear (if rollback failed - what is the state of
statement/connection/transaction?)
Reviewed-by: Preston Thorpe <preston@turso.tech>
Closes#3591
There is currently a bug found in our materialized view implementation
that happens when we delete a row, and then re-insert another row with
the same primary key.
Our insert code needs to detect updates and generate a DELETE + INSERT.
But in this case, after the initial DELETE, the fresh insert generates
another delete.
We ended up with the wrong response for aggregations (and I am pretty
sure even filter-only views would manifest the bug as well), where
groups that should still be present just disappeared because of the
extra delete.
A new test case is added that fails without the fix.
Closes#3601
There is currently a bug found in our materialized view implementation
that happens when we delete a row, and then re-insert another row with
the same primary key.
Our insert code needs to detect updates and generate a DELETE +
INSERT. But in this case, after the initial DELETE, the fresh insert
generates another delete.
We ended up with the wrong response for aggregations (and I am pretty
sure even filter-only views would manifest the bug as well), where
groups that should still be present just disappeared because of the
extra delete.
A new test case is added that fails without the fix.
I added the `Once` before so fix a bug, but it was a bit hackery. We can
`get_or_init` to achieve the same purpose, and the code becomes much
cleaner. `get_or_init` guarantees the init will happen only once as
well.
Reviewed-by: Preston Thorpe <preston@turso.tech>
Reviewed-by: bit-aloo (@Shourya742)
Closes#3578
For insert-heavy checkpoints this gives a much higher chance of using
the balance-quick subalgorithm instead of the more complex and slower
balance-nonroot.
Closes#3589
