This PR add basic CDC functionality to the `turso-db`.
### Feature components
1. `unstable_capture_data_changes_conn` pragma which allow user to turn
on/off CDC logging for **specific connection**
* CDC will have multiple modes, but for now only `off` / `rowid-
only` are supported
* Default CDC table is `turso_cdc` but user can override this with
`PRAGMA` update syntax and use arbitrary table for the CDC needs
* This can be helpful in future if turso will need to break table
format compatibility and custom tables can be a way to migrate between
different schemas
* Update syntax for the pragma accepts one string argument in
format, where only mode is set or custom cdc table name is provided as
second part of the string, separated with comma from the mode
```sql
turso> PRAGMA unstable_capture_data_changes_conn('rowid-only');
turso> PRAGMA unstable_capture_data_changes_conn('off');
turso> PRAGMA unstable_capture_data_changes_conn('rowid-only,custom_cdc_table');
turso> PRAGMA unstable_capture_data_changes_conn;
┌────────────┬──────────────────┐
│ mode │ table │
├────────────┼──────────────────┤
│ rowid-only │ custom_cdc_table │
└────────────┴──────────────────┘
```
2. CDC table schema right now is simple but it will be evolved soon to
support logging of row values before/after the change:
```sql
CREATE TABLE custom_cdc_table (
operation_id INTEGER PRIMARY KEY AUTOINCREMENT,
operation_time INTEGER, -- unixepoch() at the moment of insert, can drift if machine clocks is not monotonic
operation_type INTEGER, -- -1 = delete, 0 = update, 1 = insert
table_name TEXT,
id
)
```
* Note, that `operation_id` is marked as `AUTOINCREMENT` but `turso-
db` needs to implement
https://github.com/tursodatabase/turso/issues/1976 in order to properly
support that keyword
3. Query planner changes are made in `INSERT`/`UPDATE`/`DELETE` plans in
order to emit updates to the CDC table for changes in the table
* Note, that row `UPDATE` which change primary key generate `DELETE` +
`INSERT` statement instead of single `UPDATE`
### Implementation details
- `PRAGMA` to enable CDC is **unstable** which means that publicly
visible side-effects/public API can change in future (and it will change
soon in order to support more rich CDC modes)
- CDC table is just a regular table with its benefits and downsides:
* benefits: user can perform maintenance operations with that table
just with regular SQL like `DELETE FROM turso_cdc WHERE operation_id <
?` to cleanup old not needed CDC entries
* downsides: user can accidentally make unwanted change to CDC table
- Changes to CDC table is not logged to itself
* Note, that different connections (e.g. `C1`, `C2`) can have
different CDC tables set (e.g. `A` and `B`) - in which case changes made
to CDC table `B` through connection `C1` will be reflected in CDC table
`A`
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#1926
This implements a subset of SQLite TCL tests. Let's start fixing errors
and then import more tests!
The test cases are imported from SQLite 3.50.2.
Refs #62Closes#1980
The `.headers` command takes `on` and `off` as parameter, supported by
SQLite, which controls whether result set header is printed in list
mode.
Closes#1983
Previously all constraints were disabled with an `if false {}` hack.
In this PR:
- enable PRIMARY KEYs and UNIQUE constraints if `experimental_indexes`
compile-time feature is enabled.
- Also enable PRIMARY KEY for exactly INTEGER columns, as INTEGER
PRIMARY KEY does not require an index
- NOT NULL constraint is also now enabled in both cases, as it doesn't
require an index
Closes#1981
We're now mixing different error messages, which makes compatibility
testing pretty hard. Unify on a single, SQLite compatible error message
"no such table".
Closes#1978
We're now mixing different error messages, which makes compatibility
testing pretty hard. Unify on a single, SQLite compatible error message
"no such table".
Previously `ScalarFunc::Glob` only handled Text based values. After this
fix it will be able to handle all types of values.
Now:
```SQL
turso> CREATE TABLE IF NOT EXISTS t0 (c0 REAL );
turso> UPDATE t0 SET c0='C2IS*24', c0=0Xffffffffbfc4330f, c0=0.6463854797956918 WHERE ((((((((t0.c0)AND(t0.c0)))AND(0.23913649834358142)))OR(CASE t0.c0 WHEN t0.c0 THEN 'j2' WHEN t0.c0 THEN t0.c0 WHEN t0.c0 THEN t0.c0 END)))OR(((((((((t0.c0)AND(t0.c0)))AND(t0.c0)))OR(t0.c0)))AND(t0.c0))));
turso> INSERT INTO t0 VALUES (NULL);
INSERT INTO t0 VALUES ('0&');
UPDATE t0 SET c0=2352448 WHERE ((((t0.c0)GLOB(t0.c0))) NOT NULL);
turso> SELECT * from t0;
┌───────────┐
│ c0 │
├───────────┤
│ │
├───────────┤
│ 2352448.0 │
└───────────┘
```
Fixes: https://github.com/tursodatabase/turso/issues/1953Closes#1955
I modified the `better-sqlite3.spec.mjs` test suite to use the dual test
runner from https://github.com/tursodatabase/turso/pull/1941 and I
deleted the `limbo.spec.mjs` test suite, because it's now redundant.
I looked carefully at the diff between the two test suites to make sure
that we didn't lose any coverage with this change.
Closes#1959
