Also fixes an issue where since the old Database wasn't `Drop`d, it
caused an issue with the static Registry ™️ that had broken
`--doublecheck` mode.
Closes#2623
This continues the work for `printf` support from from tthe issue
https://github.com/tursodatabase/turso/issues/885.
This PR adds support for `%i` operands in the printf function. An
example of a command that works now and didn't before:
```sql
SELECT PRINTF("Decimals: %d %i", 200, 300);
```
This is Just an initial very simple contribution to get my feet wet and
learn the basics about the project and how contributions here work. I
intend to do something more meaningful in next contributions. Probably
will try to finish support for missing `printf` functionality also.
Closes#2615
This PR switches from usage of WAL push to SQL over http calls.
As we have sequence of logical changes from CDC - we can just send SQL
statements to the remote. This will give us benefits of more smooth
concurrent usage of sync - because WAL push is very unfriendly in case
of concurrent usages.
Closes#2617
There is a distinction between tests that verify extension-specific
behavior and those that verify interactions between the database engine
and extensions. Previously, both types of tests were kept in
`extensions.py`. With this new framework, we can extract the latter type
of tests from `extensions.py` into TCL. This cleans up `extensions.py`
and enables compatibility testing with SQLite at no extra cost.
I’m currently working on supporting outer joins involving TVFs and
planning to add more tests that exercise the database’s handling of
virtual tables, so I decided to do this refactoring first.
In the future, we may consider moving extension-specific tests to TCL as
well, especially those that have counterparts in SQLite or sqlean.
Reviewed-by: Preston Thorpe <preston@turso.tech>
Closes#2556
## 1. select equal number of columns per compound subselect in simulator
#2609 fixed compound selects incorrectly, introducing another bug: now
the sim can SELECT a different number of columns per subselect, which is
illegal. this PR forces the sim to select the same number of cols per
subselect.
Depends on #2614 , otherwise the sim constantly fails whenever it
decides to do a compound select with DISTINCT.
Closes#2611
## 2. introduce TableContext for the simulator to properly generate
predicates for Joins
The simulator has a construct called `JoinTable` which is really a
`JoinContext` that includes all the columns from the so-far joined
tables, so that the next table to be joined can refer to columns from
any of those tables. @pedrocarlo 's commit fixes an issue where
`JoinTable` would incorrectly generate predicates with empty table
names. Related: #2618Closes#2616
Closes#2612
Two commits:
## Simplify ORDER BY sorter column remapping
In case an ORDER BY column exactly matches a result column in the
SELECT,
the insertion of the result column into the ORDER BY sorter can be
skipped
because it's already necessarily inserted as a sorting column.
For this reason we have a mapping to know what index a given result
column
has in the order by sorter.
This commit makes that mapping much simpler.
## Fix DISTINCT with ORDER BY
We had a bug where we were checking for duplicates in the DISTINCT
index based on both the result column count plus any ORDER BY columns
not present in the DISTINCT clause.
This is wrong, so fix it by only using the result columns for the
dedupe check.
Closes#2614
These tests verify interactions between the database engine and TVFs.
They happen to use generate_series, but they are not intended to test
the behavior of any specific extension. Tests that verify generate_series
specific behavior remain in extensions.py.
There is a distinction between tests that verify extension-specific
behavior and tests that verify interactions between the database engine
and extensions. Previously, both types of tests were kept in extensions.py.
With this new framework, we can extract the latter type of tests from
extensions.py into TCL. This cleans up extensions.py and provides
compatibility testing with SQLite at no extra cost.
To demonstrate the framework’s usage, tests verifying the handling of
virtual tables were extracted to TCL.
In the future, we may consider moving extension-specific tests to TCL as
well, especially those that have counterparts in SQLite or sqlean.
We had a bug where we were checking for duplicates in the DISTINCT
index based on both the result column count plus any ORDER BY columns
not present in the DISTINCT clause.
This is wrong, so fix it by only using the result columns for the
dedupe check.
In case an ORDER BY column exactly matches a result column in the SELECT,
the insertion of the result column into the ORDER BY sorter can be skipped
because it's already necessarily inserted as a sorting column.
For this reason we have a mapping to know what index a given result column
has in the order by sorter.
This commit makes that mapping much simpler.
Set in-memory WAL information according to last commited frame
- before that pages_in_frames, frame_cache and last_checksum was set to
the latest written (not commited!) frame in the WAL found on disk
Simple example to reproduce the data-loss:
```sh
n=$RANDOM
./target/debug/tursodb data-loss-$n.db 'create table t(x)';
strace -o /dev/null -e fault=pwrite64:error=EIO:when=5 ./target/debug/tursodb data-loss-$n.db 'insert into t values (randomblob(4 * 4096));'
./target/debug/tursodb data-loss-$n.db 'insert into t values (1);'
./target/debug/tursodb data-loss-$n.db 'select x from t;'
```
The main bug which contributes to the data-loss was update of
`last_checksum` to the value from last frame read from WAL while
actually DB must use checksum of last **commited** frame. Due to this
bug, `insert` from third operation used checksum from last **written**
frame, but continued to append wal from the position after last
**committed** frame. So, fourth operation read WAL, detected checksum
mismatch and dropped all frames written by third operation.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#2613
We are relying too much on the currently disabled `FaultyQuery` and
`FsyncNoWait` to assert the basic invariant of "the rows we have
inserted/updated/deleted in the database are still there", so adding
that as a separate `Property` here.
Closes#2610
Problem:
sim was generating compound selects like this:
- pick a random `table`
- create a random single SELECT
- create `n` random compound SELECTs (UNION ALL etc)
assign a WHERE clause that always has a condition based on `table`
This can result in e.g.
```
SELECT a FROM foo WHERE bar.x = 'baz'
```
Solution:
Don't base the WHERE clause on a table that might not be referenced
in the query.
Again, the only reason this wasn't caught before was because `FaultyQuery`
and `FsyncNoWait` are the only paths where this is actually tested with an
assertion, and those are both disabled
Closes#2588
SQLite internally implements `.clone` by doing something like piping
`.dump` into a new connection to a database attached to the file you
want. This PR implements that by adding an `ApplyWriter` that implements
`std::fmt::Write`, and refactors our current `.dump` plumbing to work
with any `Write` interface, so we can `.dump` to stdout or `.dump` to a
new connection, therefore cloning the database.
Reviewed-by: Nikita Sivukhin (@sivukhin)
Closes#2590
