This PR implements the `Sequence` and `SequenceTest` opcodes, although
does not yet add plumbing to emit the latter.
SQLite has two distinct mechanisms that determine the final row order
with aggregates:
Traversal order of GROUP BY, and ORDER BY tiebreaking. When ORDER BY
contains only aggregate expressions and/or constants, SQLite has no
extra tiebreak key, but when ORDER BY mixes aggregate and non-aggregate
terms, SQLite adds an implicit, stable row `sequence` so “ties” respect
the input order.
This PR also fixes an issue with a query like the following:
```sql
SELECT u.first_name, COUNT(*) AS c
FROM users u
JOIN orders o ON o.user_id = u.id
GROUP BY u.first_name
ORDER BY c DESC;
```
Because ORDER BY has only an aggregate (COUNT(*) DESC) and no non-
aggregate terms, SQLite traverses the group key (u.first_name) in DESC
order in this case, so ties on c naturally appear with group keys in
descending order.
Previously tursodb would return the group key sorted in ASC order,
because it was used in all cases as the default
Closes#3287
SQLite disables the `load_extension` function by default in the core
library.
There are a couple reasons we should do the same.
1. VTab extensions need up to update the schema, and if there are
multiple connections we can/will corrupt the schema or read garbage
memory if an extension is loaded while another connection changes the
schema.
2. for _general security reasons_ ™️, which is why sqlite disables
this by default.
This PR leaves it enabled in the CLI, where there can be only a single
connection due to the lack of multi-process support right now.
Closes#3250
- Fixed some incorrect code when running interactions in differential
testing. Instead of replacing the state that was used for running the
interaction, I naively just incremented the interaction pointer.
- adjusted the comparison to check returned values without considering
the order of the rows returned
- added differential testing to run in CI
Closes#3235
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#3255
`TransitionResult::Continue` is an internal implementation detail that
tells an invocation of `StateMachine::step()` to continue looping, but
it is of no use to upstream callers.
For this reason, just return an IOResult from StateMachine::step() which
simplifies the result handling.
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#3248
TransitionResult is an internal implementation detail that tells
an invocation of StateMachine::step() to continue looping, but it
is of no use to other callers.
For this reason, just return an IOResult from StateMachine::step()
which simplifies the result handling.
This PR does not implement MVCC checkpoint yet, just adds a lock for it.
MVCC checkpoints are always TRUNCATE, plus they block all other
transactions. This guarantees that never need to let transactions read
from the SQLite WAL.
In MVCC, the checkpoint procedure is roughly as follows:
- Take the blocking_checkpoint_lock
- Write everything in the logical log to the pager, and from there
commit to the SQLite WAL.
- Immediately TRUNCATE checkpoint the WAL into the database file.
- Release the blocking_checkpoint_lock.
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#3244
This PR adds support for partial indexes, e.g. `CREATE INDEX` with a
provided predicate
```sql
CREATE UNIQUE INDEX idx_expensive ON products(sku) where price > 100;
```
The PR does not yet implement support for using the partial indexes in
the optimizer.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#3228
- Made the code around snapshot isolation more ergonomic with each
connections having its own transaction state. Also when shadowing, we
pass a ShadowTablesMut object that dynamically uses the committed tables
or the connection tables depending on the transaction state.
- added begin concurrent transaction before every property when mvcc is
enabled (this is just so we can have some mvcc code be tested using the
simulator under Begin Concurrent, I have not yet implemented the logic
to have concurrent transactions in the simulator)
- some small enhancements to shrinking
TODOs
- have proper logic to have concurrent transactions without WriteWrite
conflicts. This means when generating the plans we need to make sure
that we do not generate rows that will conflict with rows in other
transactions. This is slightly more powerful than what we do in the
fuzzer, as we just have `WriteWriteConflict` as an acceptable error
there. By baking this `NoConflict` approach to the simulator, we can
continuously test the what does not trigger a WriteWriteConflict and
snapshot isolation.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#3226
MVCC checkpoints are always TRUNCATE, plus they block all other transactions.
This guarantees that never need to let transactions read from the SQLite WAL.
In MVCC, the checkpoint procedure is roughly as follows:
- Take the blocking_checkpoint_lock
- Write everything in the logical log to the pager, and from there commit to the SQLite WAL.
- Immediately TRUNCATE checkpoint the WAL into the database file.
- Release the blocking_checkpoint_lock.
Add a `Once` object to uphold this property. We cannot use the OnceLock
to dictate this, because if we set the OnceLock before actually calling
the callback, there is a moment in time where we will have an incorrect
transient state. This change ensures we atomically call the callback and
then set the OnceLock
Should fix#3217Closes#3217Closes#3237
