This PR imports the `sync` test suite from libSQL, and modifies the
export structure match `better-sqlite3`, so that at least a few tests
from the new test suite are passing.
I also changed the `package.json` to expose `wrapper.js` as an
entrypoint. I think the plain `index.js` was probably never meant to be
exposed directly to clients, because the wrapper does some important
transformation. It's also how libSQL-js is
[structured](https://github.com/tursodatabase/libsql-
js/blob/main/package.json#L20).
## DualTest test runner
The test suite that I imported was previously run twice, with different
environment variables: one run for libSQL-js, one run for better-
sqlite3. This worked well with libSQL, because it's compatible with
better-sqlite3. But Turso isn't there yet, so even though all tests need
to run on better-sqlite3, not all tests are passing on Turso.
To make it possible to run the test suite on both implementation, and to
make it possible to track the progress of Turso, I've added a `DualTest`
test runner that emulates the API of the Ava test runner, but instead of
a single `test()` function, it exposes two methods: `both()`, and
`onlySqlitePasses()`. The first one runs the test on both
implementations, and the second one also runs it on both, but expects
that Turso will fail.
When Turso is completely compatible with better-sqlite3, it will be easy
to remove the dual-test runner, since it has the same API as Ava.
## Future development
### Existing tests
The existing tests were divided in two files: `better-sqlite3.spec.mjs`,
and `dual-test.mjs` that are kept in sync manually. The first one is
mostly a superset of the second one, with additional tests indicating
behaviour that isn't implemented in Turso yet. I want to merge these
test suites to also use the dual-test test runner. This will make it
easier to evolve test suites and to track the progress of Turso.
### `SqliteError`
Modifying the test called `errors` to `both()` will show that Turso is
still missing an `SqliteError` type, to be compatible with better-
sqlite3. I have the required changes in a stash and will open a PR
shortly.
-----
as part of https://github.com/tursodatabase/turso/issues/1900Closes#1941
Added a threshold to clip input page cache size values to `0`. This
avoid the panic at runtime and also follow's SQLite behavior closely.
Now:
```
Turso v0.1.1
Enter ".help" for usage hints.
This software is ALPHA, only use for development, testing, and experimentation.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database
turso> PRAGMA main.cache_size = 5022422913188235998;
turso> PRAGMA main.cache_size;
┌────────────┐
│ cache_size │
├────────────┤
│ 10 │
└────────────┘
```
Closes: https://github.com/tursodatabase/turso/issues/1952Closes#1954
As adaptive colors use `OSC 11` to query the terminal's background
color, the query string may contaminate test result and cause failures
when running tests in a terminal.
```
make test-vector
SQLITE_EXEC=scripts/limbo-sqlite3 ./testing/vector.test
(testing/testing.db) Running test: vector-functions-valid
Test FAILED: SELECT vector_extract(vector('[]'));
SELECT vector_extract(vector(' [ 1 , 2 , 3 ] '));
SELECT vector_extract(vector('[-1000000000000000000]'));
returned '[]
[1,2,3]
[-1000000000000000000]'
expected '[]
[1,2,3]
[-1000000000000000000]'
^[]11;rgb:158e/193a/1e75^[\make: *** [test-vector] Error 1
```
Closes#1942
Fixes#1890
Once rollback was implement we quickly saw that it lacked support for
schema changes so we had to re-estructure things a bit.
## Example of failure:
```bash
turso> begin;
turso> create table t(x);
turso> rollback;
turso> pragma integrity_check;
thread 'main' panicked at core/storage/sqlite3_ondisk.rs:386:36:
called `Result::unwrap()` on an `Err` value: Corrupt("Invalid page type: 83")
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
This happened because it thought table `t` existed because we didn't
rollback that schema.
## Changes:
* The most important change: now every connection has a private copy of
schema. On write txn commit we update a global schema shared between
connections in order for new connections to get updated version from
there. In case of rollback, we simply change connection's schema to
previous version. This change allowed us to remove locks for schema
private copy and keeping schema changes locally in case of concurrency.
Sqlite does things differently, they lazily parse schema in case of
outdated schema, this many schema changes to trigger reading schema from
db file which is slow. If we are able to keep local copy in memory, even
when if we add multiprocessing, it will speed up schema reloading by a
bunch.
* `schema_cookie` is now update for every schema change
* `Insn::ParseSchema` had a nasty bug where it would commit all the
changes made in a query that changed a schema, we fixed that by setting
`auto_commit` to `false` before parsing schema, and setting it back to
previous value once schema is parsed.
Closes#1928
This PR will close#1446. Buffer pool implementation should be thread
safe after this PR.
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#1910
When an `UPDATE` statement modifies a table's `INTEGER PRIMARY KEY`
(which acts as a `rowid` alias) alongside other indexed columns, the
index entries were incorrectly retaining the old `rowid`. This led to
stale index references, causing subsequent queries to return incorrect
results.
This change ensures that when the `rowid` alias is part of the `SET`
clause in an `UPDATE` statement, the new `rowid` value is used for
generating and updating index records. This guarantees that all index
entries correctly point to the updated row, resolving the data
inconsistency.
Fixes#1897Closes#1916
When we renamed the Python imports, we forgot to rename the
`test_limbo_cli` to `test_turso_cli`. Also adds a vfs-bench-compile
github ci to compile and test that the bench-vfs scripts works
Closes#1933
