Commit Graph

1156 Commits

Author SHA1 Message Date
meteorgan
c6ef4898b0 fix: IdxDelete shouldn't raise error if P5 == 0 2025-07-08 22:57:20 +08:00
meteorgan
4a516ab414 Support except operator for compound select 2025-07-08 22:57:20 +08:00
Jussi Saurio
cb8a576501 op_idx_insert: introduce flag for ignoring duplicates 2025-07-08 12:14:20 +03:00
Pekka Enberg
84771f17f7 Merge 'core/translate: Fix aggregate star error handling in prepare_one_sele…' from Pekka Enberg
…ct_plan()
For example, if we attempt to do `max(*)`, let's return the error
message from `resolve_function()` to be compatible with SQLite:
```
sqlite> CREATE TABLE test1(f1, f2);
sqlite> SELECT max(*) FROM test1;
Parse error: wrong number of arguments to function max()
  SELECT max(*) FROM test1;
         ^--- error here
```
Spotted by SQLite TCL tests.

Closes #1990
2025-07-08 10:19:59 +03:00
Pekka Enberg
341f963a8e Merge 'Fix infinite loops, rollback problems, and other bugs found by I/O fault injection' from Pedro Muniz
Was running the sim with I/O faults enabled and fixed some nasty bugs.
Now, there are some more nasty bugs to fix as well. This is the command
that I use to run the simulator `cargo run -p limbo_sim -- --minimum-
tests 10 --maximum-tests 1000`
This PR mainly fixes the following bugs:
- Not decrementing in flight write counter when `pwrite` fails
- not rolling back the transaction on `step` error
- not rolling back the transaction on `run_once` error
- some functions were just being unwrapped when they could suffer io
errors
- Only change max_frame after wal sync's

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1946
2025-07-07 21:31:26 +03:00
Pekka Enberg
d4c03d426c core/translate: Fix aggregate star error handling in prepare_one_select_plan()
For example, if we attempt to do `max(*)`, let's return the error
message from `resolve_function()` to be compatible with SQLite:

```
sqlite> CREATE TABLE test1(f1, f2);
sqlite> SELECT max(*) FROM test1;
Parse error: wrong number of arguments to function max()
  SELECT max(*) FROM test1;
         ^--- error here
```

Spotted by SQLite TCL tests.
2025-07-07 19:56:09 +03:00
Pekka Enberg
0d17d35ef4 Merge 'Change data capture' from Nikita Sivukhin
This PR add basic CDC functionality to the `turso-db`.
### Feature components
1. `unstable_capture_data_changes_conn` pragma which allow user to turn
on/off CDC logging for **specific connection**
    * CDC will have multiple modes, but for now only `off` / `rowid-
only` are supported
    * Default CDC table is `turso_cdc` but user can override this with
`PRAGMA` update syntax and use arbitrary table for the CDC needs
      * This can be helpful in future if turso will need to break table
format compatibility and custom tables can be a way to migrate between
different schemas
    * Update syntax for the pragma accepts one string argument in
format, where only mode is set or custom cdc table name is provided as
second part of the string, separated with comma from the mode
```sql
turso> PRAGMA unstable_capture_data_changes_conn('rowid-only');
turso> PRAGMA unstable_capture_data_changes_conn('off');
turso> PRAGMA unstable_capture_data_changes_conn('rowid-only,custom_cdc_table');
turso> PRAGMA unstable_capture_data_changes_conn;
┌────────────┬──────────────────┐
│ mode       │ table            │
├────────────┼──────────────────┤
│ rowid-only │ custom_cdc_table │
└────────────┴──────────────────┘
```
2. CDC table schema right now is simple but it will be evolved soon to
support logging of row values before/after the change:
```sql
CREATE TABLE custom_cdc_table (
  operation_id INTEGER PRIMARY KEY AUTOINCREMENT,
  operation_time INTEGER, -- unixepoch() at the moment of insert, can drift if machine clocks is not monotonic
  operation_type INTEGER, -- -1 = delete, 0 = update, 1 = insert
  table_name TEXT,
  id
)
```
  * Note, that `operation_id` is marked as `AUTOINCREMENT` but `turso-
db` needs to implement
https://github.com/tursodatabase/turso/issues/1976 in order to properly
support that keyword
3. Query planner changes are made in `INSERT`/`UPDATE`/`DELETE` plans in
order to emit updates to the CDC table for changes in the table
  * Note, that row `UPDATE` which change primary key generate `DELETE` +
`INSERT` statement instead of single `UPDATE`
### Implementation details
- `PRAGMA` to enable CDC is **unstable** which means that publicly
visible side-effects/public API can change in future (and it will change
soon in order to support more rich CDC modes)
- CDC table is just a regular table with its benefits and downsides:
  * benefits: user can perform maintenance operations with that table
just with regular SQL like `DELETE FROM turso_cdc WHERE operation_id <
?` to cleanup old not needed CDC entries
  * downsides: user can accidentally make unwanted change to CDC table
- Changes to CDC table is not logged to itself
  * Note, that different connections (e.g. `C1`, `C2`) can have
different CDC tables set (e.g. `A` and `B`) - in which case changes made
to CDC table `B` through connection `C1` will be reflected in CDC table
`A`

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1926
2025-07-07 18:03:07 +03:00
pedrocarlo
b85687658d change instrumentation level to INFO 2025-07-07 11:53:45 -03:00
pedrocarlo
5559c45011 more instrumentation + write counter should decrement if pwrite fails 2025-07-07 11:50:21 -03:00
pedrocarlo
897426a662 add error tracing to relevant functions + rollback transaction in step_end_write_txn + make move_to_root return result 2025-07-07 11:50:21 -03:00
Nikita Sivukhin
1655c0b84f small fixes 2025-07-07 12:50:10 +04:00
Pekka Enberg
7f91768ff6 core/translate: Unify no such table error messages
We're now mixing different error messages, which makes compatibility
testing pretty hard. Unify on a single, SQLite compatible error message
"no such table".
2025-07-07 11:10:46 +03:00
Nikita Sivukhin
1ee475f04a rename pragma to unsable_capture_data_changes_conn 2025-07-06 22:32:42 +04:00
Nikita Sivukhin
a10d423aac adjust schema 2025-07-06 22:30:57 +04:00
Nikita Sivukhin
62c1e38805 small fixes 2025-07-06 22:26:34 +04:00
Nikita Sivukhin
32fa2ac3ee avoid capturing changes in cdc table 2025-07-06 22:24:35 +04:00
Nikita Sivukhin
a988bbaffe allow to specify table in the capture_data_changes PRAGMA 2025-07-06 22:19:32 +04:00
Nikita Sivukhin
271b8e5bcd fix clippy 2025-07-06 21:16:58 +04:00
Nikita Sivukhin
40769618c1 small refactoring 2025-07-06 21:16:58 +04:00
Nikita Sivukhin
04f2efeaa4 small renames 2025-07-06 21:16:57 +04:00
Nikita Sivukhin
a82529f55a emit cdc changes for UPDATE / DELETE statements 2025-07-06 21:16:25 +04:00
Nikita Sivukhin
d72ba9877a emit turso_cdc table changes in Insert query plan 2025-07-06 21:16:25 +04:00
Nikita Sivukhin
234dda322f handle change_capture pragma 2025-07-06 21:16:25 +04:00
Nikita Sivukhin
b0fc67a314 pass ownership or program to the pragma translators - just as with other statements 2025-07-06 21:16:25 +04:00
Krishna Vishal
19d949521a Add a threshold to clip large page cache values to 0. This prevents
panic at runtime.
2025-07-04 10:24:10 +05:30
Pere Diaz Bou
470fb8d23b rollabck translate remove querymode 2025-07-03 12:36:48 +02:00
Pere Diaz Bou
cde7202981 Revert "Merge 'core: Disable ROLLBACK statement' from Pekka Enberg"
This reverts commit 8a13e4b02f, reversing
changes made to cc935f97cc.
2025-07-03 12:36:48 +02:00
Pere Diaz Bou
f37893eb8f set cookie for index operations 2025-07-03 12:36:48 +02:00
Pere Diaz Bou
ba988685cf set cookie create virtual table 2025-07-03 12:36:48 +02:00
Pere Diaz Bou
d8658264d9 alter set cookie 2025-07-03 12:36:48 +02:00
Pere Diaz Bou
c799396c3d rollback schema in connection 2025-07-03 12:36:48 +02:00
Pere Diaz Bou
5b733663ab update schema in case it's outdated 2025-07-03 12:36:48 +02:00
Pere Diaz Bou
abf1699dd2 set scheam version and update shared schema in txn 2025-07-03 12:36:48 +02:00
Pekka Enberg
471d26a632 Merge 'Fix index update when INTEGER PRIMARY KEY (rowid alias)' from Adrian-Ryan Acala
When an `UPDATE` statement modifies a table's `INTEGER PRIMARY KEY`
(which acts as a `rowid` alias) alongside other indexed columns, the
index entries were incorrectly retaining the old `rowid`. This led to
stale index references, causing subsequent queries to return incorrect
results.
This change ensures that when the `rowid` alias is part of the `SET`
clause in an `UPDATE` statement, the new `rowid` value is used for
generating and updating index records. This guarantees that all index
entries correctly point to the updated row, resolving the data
inconsistency.
Fixes #1897

Closes #1916
2025-07-03 13:10:53 +03:00
Nikita Sivukhin
c9c5ef4e25 remote query_mode from ProgramBuilderOpts and from function arguments
- mode never changes and ProgramBuilder already created with proper mode set correctly
2025-07-02 13:24:12 +04:00
AdrianAcala
7ca902979d Fix: Correctly update indexes when INTEGER PRIMARY KEY (rowid alias) changes (Issue #1897)
When an `UPDATE` statement modifies a table's `INTEGER PRIMARY KEY` (which acts as a `rowid` alias) alongside other indexed columns, the index entries were incorrectly retaining the old `rowid`. This led to stale index references, causing subsequent queries to return incorrect results.

This change ensures that when the `rowid` alias is part of the `SET` clause in an `UPDATE` statement, the new `rowid` value is used for generating and updating index records. This guarantees that all index entries correctly point to the updated row, resolving the data inconsistency.
2025-07-01 16:03:27 +00:00
Levy A.
25927d91d8 cargo fmt 2025-06-30 14:37:51 -03:00
Levy A.
ffd6844b5b refactor: remove PseudoTable from Table
the only reason for `PseudoTable` to exist, is to provide column
information for `PseudoCursor` creation. this should not be part of the
schema.
2025-06-30 14:31:58 -03:00
Levy A.
3907b387b3 cargo fix 2025-06-30 14:01:47 -03:00
Levy A.
afc55b27f0 refactor: remove unnecessary column definitions for PseudoTable
the only information that matters in the amount of column
2025-06-30 13:54:29 -03:00
Pekka Enberg
2542cb2d03 core: Disable ROLLBACK statement
There's bad interaction with schema changes and `ROLLBACK`:

https://github.com/tursodatabase/turso/issues/1890

Disable the statement for now to avoid people hitting the issue.
2025-06-30 17:30:01 +03:00
Pekka Enberg
b87ce6d178 Merge 'Fix deleting previous rowid when rowid is in the Set Clause' from Pedro Muniz
Closes #1888 . This PR fixes UPDATE translation by not emitting an
ephemeral plan when we are doing a `RowIdEq` search. Also, we should
delete the previous rowid when the rowid is in the set clause.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1891
2025-06-30 11:58:05 +03:00
Pekka Enberg
9c1b7897ac Fix URLs to point to github.com/tursodatabase/turso 2025-06-30 11:23:53 +03:00
pedrocarlo
7e0225b1af add some comments 2025-06-29 17:37:46 -03:00
pedrocarlo
738e2cc06c do not emit ephemeral plan when doing a SeekRowId + emit Delete instruction when rowid in set clause 2025-06-29 17:12:24 -03:00
Pekka Enberg
725c3e4ddc Rename limbo_sqlite3_parser crate to turso_sqlite3_parser 2025-06-29 12:34:46 +03:00
Pekka Enberg
eb0de4066b Rename limbo_ext crate to turso_ext 2025-06-29 12:14:08 +03:00
Pekka Enberg
09ba89e2ba core/translate: Replace todo with bail_parse_error
No point in crashing the whole app if someone attempts to change page
size.
2025-06-27 13:42:49 +03:00
Pekka Enberg
c12b291f9a Merge 'Fix evaluation of ISNULL/NOTNULL in OR expressions' from Piotr Rżysko
Previously, the `jump_if_condition_is_true` flag was not respected. As a
result, for expressions like <`ISNULL`/`NOTNULL`> `OR` <rhs>, the <rhs>
expression was evaluated even when the left-hand side was true, and its
value was incorrectly used as the final result.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1846
2025-06-27 13:15:21 +03:00
Piotr Rzysko
116df2ec86 Fix evaluation of ISNULL/NOTNULL in OR expressions
Previously, the `jump_if_condition_is_true` flag was not respected. As a
result, for expressions like <`ISNULL`/`NOTNULL`> `OR` <rhs>, the <rhs>
expression was evaluated even when the left-hand side was true, and its
value was incorrectly used as the final result.
2025-06-27 08:21:40 +02:00