#560
Changes to `translate_expr` function:
* [`core/translate/expr.rs`](diffhunk://#diff-
371865d5d7b8bcaed649413c687492e61e94f21387cd9b2c47d989a033888c8bL1558-
R1560): Changed the `amount` parameter in the `Insn::Copy` instruction
from `1` to `0`.
Enhancements to the testing framework:
* [`testing/scalar-functions.test`](diffhunk://#diff-
a046d58ab24eee8207f0ce3199f8d0a609edcef9c24b8ed7f242f7a60e6c1e61R812-
R815): Added a new test `do_execsql_test_regex` to validate that the
`sqlite_version` function returns a valid output.
* [`testing/tester.tcl`](diffhunk://#diff-
316cca92d85df3f78558cc3e60d7420c1fd19a23ecf2bbea534db93ab08ea3ecR29-
R45): Introduced a new procedure `do_execsql_test_regex` to support
regex-based validation of SQL outputs.
Closes#561
### Purpose of this PR
Support for DELETE query execution planning
### Implementation Details
The main entry point for planning DELETE queries is the
`translate_delete` function. It is composed of three primary steps:
- `prepare_delete_plan`:
- Reuses the existing SELECT query's WHERE clause parsing logic to
interpret and construct the initial delete plan.
- `optimize_delete_plan`:
- eliminating BETWEEN expressions
- usage of indexes
- `emit_program_for_delete`:
- Add instructions for delete operation
I've tried to reuse existing logic(mostly on SELECT operations) as much
as I can, so that we can automatically incorporate new changes
automatically.
### Delete planning debug
I've used `println!(...)` to specify the rows to delete. Example below
<img width="374" alt="image" src="https://github.com/user-
attachments/assets/f109e1c6-6b69-43b9-bb23-4bee3a835767" />
### Bytecode compatibility
`EXPLAIN DELETE FROM users WHERE id = 1;`
<img width="1724" alt="image" src="https://github.com/user-
attachments/assets/ce2995d7-6947-493e-ad3d-224df7f4e7c2" />
`EXPLAIN DELETE FROM users WHERE id > 3`
<img width="1726" alt="image" src="https://github.com/user-
attachments/assets/ac516bd2-fe80-44c5-9a4b-8e35d574c47d" />
`EXPLAIN DELETE FROM users WHERE id < 3`
<img width="1711" alt="image" src="https://github.com/user-
attachments/assets/29d0ccba-c373-483e-bb6b-9344289cae02" />
### TODO(future works)
- Add support for `Clear` opcode
- Add test when `delete` is implemented in `Cursor`
<img width="1728" alt="image" src="https://github.com/user-
attachments/assets/28d371fc-90f5-42e5-8add-d5218f830234" />
Closes#538
The name "row result" is confusing because it really *is* a result from
a step() call. The only difference is how a row is represented as we
return from VDBE or from a statement.
Therefore, rename RowResult to StepResult.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#558
The name "row result" is confusing because it really *is* a result from
a step() call. The only difference is how a row is represented as we
return from VDBE or from a statement.
Therefore, rename RowResult to StepResult.
In line with [other work](#127) for JSON support, this PR adds support
for [`json_array_length`](https://www.sqlite.org/json1.html#jarraylen).
This includes a first pass at supporting the JSON path for accessing
values within the JSON.
I've added tests in rust and tcl.
Closes#555
This PR's genesis is from investigating #532, but I still can't reliably
reproduce it on either `main` or this branch so I don't know if this PR
_fixes_ anything, but I guess it aligns us more with sqlite anyway
---
Anyway: I looked at DBs created with limbo and with sqlite using
[ImHex](https://github.com/WerWolv/ImHex) and the differences seem to
be:
1. SQLite uses varint according to [the
spec](https://www.sqlite.org/fileformat.html#record_format), whereas
limbo always encodes integers as i64
2. Limbo adds 4 bytes of zeros for overflow page pointer (even in cases
where the cell doesnt overflow)
3. Limbo adds a space after `CREATE TABLE name` before the `(` even when
user doesn't specify it?
I implemented the following:
- Fix 1: Varint serialization of i8, i16, i24, i32, i48 and i64
according to payload, instead of always using i64
- Fix 2: Removed the 4 bytes reserved for overflow page pointer in non-
overflow cases
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#550
small follow up to https://github.com/tursodatabase/limbo/pull/539
contains:
- Variable renaming and comments to `btreecursor.insert_into_cell()`
- New utility methods `pagecontent.header_size()`,
`pagecontent.cell_pointer_array_size()`,
`pagecontent.unallocated_region_start()` and
`pagecontent.unallocated_region_size()`
- Refactor of `btreecursor.compute_free_space()` (plus comments and
variable renaming)
- Rename `pagecontent.cell_get_raw_pointer_region()` to
`pagecontent.cell_pointer_array_offset_and_size()` and remove its usage
in `btreecursor.defragment_page()`
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#543
Closes#436
This PR fixes four issues:
1. Not respecting user-provided column names (e.g. `INSERT INTO foo
(b,c) values (1,2);` would just insert into the first two columns
regardless of what index `b` and `c` have)
2. Limbo would get in an infinite loop when inserting too many values
(too many i.e. more columns than the table has)
3. False positive unique constraint error on non-primary key columns
when inserting multiple values, e.g.
```
limbo> create table t1(v1 int);
limbo> insert into t1 values (1),(2);
Runtime error: UNIQUE constraint failed: t1.v1 (19)
```
as seen [here](https://github.com/tursodatabase/limbo/pull/490#issuecomm
ent-2545545562)
4. Limbo no longer uses a coroutine for INSERT when only inserting one
row. See [this comment](https://github.com/tursodatabase/limbo/issues/43
6#issuecomment-2533937845). For the equivalent query, Limbo now
generates:
```
limbo> EXPLAIN INSERT INTO users (name, email) VALUES ('John Doe', 'john@example.com');
addr opcode p1 p2 p3 p4 p5 comment
---- ----------------- ---- ---- ---- ------------- -- -------
0 Init 0 10 0 0 Start at 10
1 OpenWriteAsync 0 2 0 0
2 OpenWriteAwait 0 0 0 0
3 String8 0 3 0 John Doe 0 r[3]='John Doe'
4 String8 0 4 0 john@example.com 0 r[4]='john@example.com'
5 NewRowId 0 1 0 0
6 MakeRecord 2 3 5 0 r[5]=mkrec(r[2..4])
7 InsertAsync 0 5 1 0
8 InsertAwait 0 0 0 0
9 Halt 0 0 0 0
10 Transaction 0 1 0 0
11 Null 0 2 0 0 r[2]=NULL
12 Goto 0 1 0 0
```
---
Note that this PR doesn't fix e.g. #472 which requires creating an index
on the non-rowid primary key column(s), nor does it implement rollback
(e.g. inserting two rows where one fails to unique constraint still
inserts the other row)
---
**EXAMPLES OF ERRONEOUS BEHAVIOR -- current head of main:**
wrong column inserted
```
limbo> create table rowidalias_b (a, b INTEGER PRIMARY KEY, c, d);
limbo> insert into rowidalias_b (d) values ('d only');
limbo> select * from rowidalias_b;
d only|1|| <-- gets inserted into column a
```
wrong column inserted
```
limbo> create table textpk (a, b text primary key, c);
limbo> insert into textpk (a,b,c) values ('a','b','c');
limbo> select * from textpk;
a|b|c
limbo> insert into textpk (b,c) values ('b','c');
limbo> select * from textpk;
a|b|c
b|c| <--- b gets inserted into column a
```
false positive from integer check due to attempting to insert wrong
column
```
limbo> create table rowidalias_b (a, b INTEGER PRIMARY KEY, c, d);
limbo> insert into rowidalias_b (a,c) values ('lol', 'bal');
Parse error: MustBeInt: the value in the register is not an integer <-- tries to insert c into b column
```
false positive from integer check due to attempting to insert wrong
column
```
limbo> CREATE TABLE users (
id INTEGER PRIMARY KEY,
name TEXT,
email TEXT
);
limbo> INSERT INTO users (name, email) VALUES ('John Doe', 'john@example.com');
Parse error: MustBeInt: the value in the register is not an integer. <-- tries to insert name into id column
```
allows write of nonexistent column
```
limbo> create table a(b);
limbo> insert into a (nonexistent_col) values (1);
limbo> select * from a;
1
```
hangs forever when inserting too many values
```
limbo> create table a (b integer primary key);
limbo> insert into a values (1,2); <-- spinloops forever at 100% cpu
```
unique constraint error on non-unique column
```
limbo> create table t1(v1 int);
limbo> insert into t1 values (1),(2);
Runtime error: UNIQUE constraint failed: t1.v1 (19)
```
**EXAMPLES OF CORRECT BEHAVIOR -- this branch:**
correct column inserted
```
limbo> create table rowidalias_b (a, b INTEGER PRIMARY KEY, c, d);
limbo> insert into rowidalias_b (d) values ('d only');
limbo> select * from rowidalias_b;
|1||d only
```
correct column inserted
```
limbo> create table textpk (a, b text primary key, c);
limbo> insert into textpk (a,b,c) values ('a','b','c');
limbo> select * from textpk;
a|b|c
limbo> insert into textpk (b,c) values ('b','c');
limbo> select * from textpk;
a|b|c
|b|c
```
correct columns inserted, PK autoincremented
```
limbo> create table rowidalias_b (a, b INTEGER PRIMARY KEY, c, d);
limbo> insert into rowidalias_b (a,c) values ('lol', 'bal');
limbo> select * from rowidalias_b;
lol|1|bal|
```
correct column inserted, PK autoincremented
```
limbo> CREATE TABLE users (
id INTEGER PRIMARY KEY,
name TEXT,
email TEXT
);
limbo> INSERT INTO users (name, email) VALUES ('John Doe', 'john@example.com');
limbo> select * from users;
1|John Doe|john@example.com
```
reports parse error correctly about wrong number of values
```
limbo> create table a (b integer primary key);
limbo> insert into a values (1,2);
Parse error: table a has 1 columns but 2 values were supplied
```
reports parse error correctly about nonexistent column
```
limbo> create table a(b);
limbo> insert into a (nonexistent_col) values (1);
Parse error: table a has no column named nonexistent_col
```
no unique constraint error on non-unique column
```
limbo> create table t1(v1 int);
limbo> insert into t1 values (1),(2);
limbo> select * from t1;
1
2
```
**Also, added multi-row inserts to simulator and ran into at least
this:**
```
Seed: 9444323279823516485
path to db '"/var/folders/qj/r6wpj6657x9cj_1jx_62cpgr0000gn/T/.tmpcYczRv/simulator.db"'
Initial opts SimulatorOpts { ticks: 3474, max_connections: 1, max_tables: 79, read_percent: 61, write_percent: 12, delete_percent: 27, max_interactions: 2940, page_size: 4096 }
thread 'main' panicked at core/storage/sqlite3_ondisk.rs:332:36:
called `Result::unwrap()` on an `Err` value: Corrupt("Invalid page type: 83")
```
Closes#533
This pr adds support for multiple readers and a single writer with a
custom made lock called `LimboRwLock`. Basically there are 5 allowed
read locks which store the max frame allowed in that "snapshot" and any
reader will try to acquire the biggest one possible. Writer will just
try to lock the `write_lock` and if not successful, it will return busy.
The only checkpoint mode supported for now is `PASSIVE` but it should be
trivial to add more modes.
This needs testing, but I will do it in another PR. I just wanted to do
it in another PR.
Closes#544
This PR makes two small incremental updates:
1- It adds a Clap CLI for simulator configuration, using the same Clap
version as the Limbo cli crate
2- It creates a new submodule called `simulator`, moving simulator
related structs from the large main file into their own files.
I am open to suggestions on the submodule name instead of `simulator` as
it's kind of weird to have `simulator/simulator` in the file tree.
Closes#540
This PR should have no functional changes, just variable renaming and
comments
Using `///` comment format for better IDE support
Reviewed-by: Pere Diaz Bou <penberg@iki.fi>
Closes#539
