Pekka Enberg
b6718471ce
Merge 'Fix EXPLAIN to be case insensitive' from Pedro Muniz
...
Closes #1286
Closes #1287
2025-04-09 20:23:09 +03:00
pedrocarlo
94217319a2
Fix Explain to be case insensitive
2025-04-09 14:21:18 -03:00
Pekka Enberg
e21e2b5449
Merge 'core: Fix syscall VFS on Linux' from Pekka Enberg
...
Fix the syscall VFS on Linux not to use `PlatformIO`, which is just an
alias for `io_uring`.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com >
Closes #1285
2025-04-09 19:38:45 +03:00
Pekka Enberg
2d009083ba
core: Fix syscall VFS on Linux
...
Fix the syscall VFS on Linux not to use `PlatformIO`, which is just an
alias for `io_uring`.
2025-04-09 19:27:58 +03:00
Pekka Enberg
a6e32c95d7
Merge 'Support UPDATE for virtual tables' from Preston Thorpe
...
closes #1144
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com >
Closes #1169
2025-04-09 18:21:10 +03:00
PThorpe92
f223e66c82
Remove unused mut and fix merge conflict issues
2025-04-09 11:15:04 -04:00
PThorpe92
13ae19c78c
Remove unnecessary clones from mc cursors
2025-04-09 11:15:04 -04:00
PThorpe92
62d1447cd6
Adapt query plan to handle vatbs for updates
2025-04-09 11:15:02 -04:00
PThorpe92
0ffecb3021
Add comments to document update on vtabs
2025-04-09 11:06:41 -04:00
PThorpe92
7993857020
Add py tests for vtab update behavior
2025-04-09 11:06:41 -04:00
PThorpe92
b685086cad
Support UPDATE for virtual tables
2025-04-09 11:06:41 -04:00
PThorpe92
2d7a27fbfa
Prevent panic in extension by out of bounds cursor idx
2025-04-09 11:06:40 -04:00
Pekka Enberg
0336783831
Merge 'Add .timer command to print SQL execution statistics' from Pere Diaz Bou
...
```
Limbo v0.0.19-pre.4
Enter ".help" for usage hints.
limbo> .timer on
limbo> select count(1) from users;
┌───────────┐
│ count (1) │
├───────────┤
│ 10000 │
└───────────┘
Command stats:
----------------------------
total: 35 ms (this includes parsing/coloring of cli app)
query execution stats:
----------------------------
Execution: avg=16 us, total=33 us
I/O: avg=123 ns, total=3 us
limbo> select 1;
┌───┐
│ 1 │
├───┤
│ 1 │
└───┘
Command stats:
----------------------------
total: 282 us (this includes parsing/coloring of cli app)
query execution stats:
----------------------------
Execution: avg=2 us, total=4 us
I/O: No samples available
```
Closes #1281
2025-04-09 17:58:48 +03:00
Pere Diaz Bou
2316d7ebf1
add .timer command with fine grained statistics about limbo
...
```
Limbo v0.0.19-pre.4
Enter ".help" for usage hints.
limbo> .timer on
limbo> select count(1) from users;
┌───────────┐
│ count (1) │
├───────────┤
│ 10000 │
└───────────┘
Command stats:
----------------------------
total: 35 ms (this includes parsing/coloring of cli app)
query execution stats:
----------------------------
Execution: avg=16 us, total=33 us
I/O: avg=123 ns, total=3 us
limbo> select 1;
┌───┐
│ 1 │
├───┤
│ 1 │
└───┘
Command stats:
----------------------------
total: 282 us (this includes parsing/coloring of cli app)
query execution stats:
----------------------------
Execution: avg=2 us, total=4 us
I/O: No samples available
```
2025-04-09 16:31:08 +02:00
Pekka Enberg
ddc5e49451
Merge 'Index insert fixes' from Pere Diaz Bou
...
Closes #1279
2025-04-09 17:21:53 +03:00
Pekka Enberg
5de2d91d04
Update COMPAT.md
2025-04-09 17:07:24 +03:00
Pekka Enberg
dbb346ba28
Update COMPAT.md
2025-04-09 17:03:53 +03:00
Pekka Enberg
3b98675aa0
Update COMPAT.md
2025-04-09 17:02:25 +03:00
Pere Diaz Bou
6a02730c1a
rebase fixes
2025-04-09 15:56:04 +02:00
Pekka Enberg
a4d9f70ef8
Merge 'Strict table support' from Ihor Andrianov
...
Closes #884
Support for
```CREATE TABLE test(id INTEGER) STRICT;```
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com >
Closes #1268
2025-04-09 16:45:56 +03:00
Pekka Enberg
3025b71406
Merge 'Memory tests to track large blob insertions' from Pedro Muniz
...
My PR #972 always keeps breaking when there are changes in how Btree
works lol. I'm extracting the memory tests I created there here, so we
can track problems earlier.
Closes #1228
2025-04-09 16:45:06 +03:00
Pere Diaz Bou
7b384f8e5c
set iteration_state for insert
2025-04-09 15:29:06 +02:00
Pekka Enberg
e67f56bb3d
Merge 'Setup tracing to allow output during test runs' from Preston Thorpe
...
## The problem:
Sometimes you want to run the tests _and_ look at logs, and previously
that wasn't possible because all the tests relied on stdout/stderr.
## The solution:
Setup `tracing-appender` to automatically log to a designated
`testing/test.log` when `RUST_LOG` is set.
This PR also starts a `testing.md` file describing testing in limbo. My
hope is that people who know stuff about DST and the fuzzing setup will
fill in the TODOs :)
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com >
Closes #1277
2025-04-09 16:16:26 +03:00
Pere Diaz Bou
6b7575bf3f
fix tree traversal assumptions on traversal
2025-04-09 15:04:45 +02:00
Pere Diaz Bou
f2d9e1e8f5
fix divider cell in index
2025-04-09 15:04:45 +02:00
Pere Diaz Bou
12899034c9
make insert idx re-entrant
2025-04-09 15:04:45 +02:00
Pere Diaz Bou
d9453f6e06
fix cell_get_raw_region length calculation
2025-04-09 15:04:45 +02:00
Pere Diaz Bou
0f59fc7e36
Merge 'Decrease page count on balancing fixes' from Pere Diaz Bou
...
* Comment how decrease of pages happen while balancing
* Free pages no longer used after balancing finished.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com >
Closes #1278
2025-04-09 15:04:25 +02:00
Pere Diaz Bou
f1df09ffd9
free no longer used pages after balance
2025-04-09 11:12:39 +02:00
Jussi Saurio
aa6e2d853a
Merge 'Support backwards index scan and seeks + utilize indexes in removing ORDER BY' from Jussi Saurio
...
## Main stuff
- Support iterating an index backwards
- Support scanning an index (instead of seeking with a condition)
- Support backwards index seeks
- Support backwards rowid seeks
- Fix existing backwards iteration logic for table btrees
- Remove ORDER BY entirely if any index satisfies the ordering
- Add fuzz tests for rowid seeks, 1 and 2 column index seeks
## Bytecode examples (note the lack of order by sorting):
one column index order by, forwards:
```sql
limbo> explain select first_name from users order by age;
addr opcode p1 p2 p3 p4 p5 comment
---- ----------------- ---- ---- ---- ------------- -- -------
0 Init 0 13 0 0 Start at 13
1 OpenReadAsync 0 2 0 0 table=users, root=2
2 OpenReadAwait 0 0 0 0
3 OpenReadAsync 1 274 0 0 table=age_idx, root=274
4 OpenReadAwait 0 0 0 0
5 RewindAsync 1 0 0 0
6 RewindAwait 1 12 0 0 Rewind table age_idx
7 DeferredSeek 1 0 0 0
8 Column 0 1 1 0 r[1]=users.first_name
9 ResultRow 1 1 0 0 output=r[1]
10 NextAsync 1 0 0 0
11 NextAwait 1 7 0 0
12 Halt 0 0 0 0
13 Transaction 0 0 0 0 write=false
14 Goto 0 1 0 0
```
one column index order by, backwards:
```sql
limbo> explain select first_name from users order by age desc;
addr opcode p1 p2 p3 p4 p5 comment
---- ----------------- ---- ---- ---- ------------- -- -------
0 Init 0 13 0 0 Start at 13
1 OpenReadAsync 0 2 0 0 table=users, root=2
2 OpenReadAwait 0 0 0 0
3 OpenReadAsync 1 274 0 0 table=age_idx, root=274
4 OpenReadAwait 0 0 0 0
5 LastAsync 1 0 0 0
6 LastAwait 1 0 0 0
7 DeferredSeek 1 0 0 0
8 Column 0 1 1 0 r[1]=users.first_name
9 ResultRow 1 1 0 0 output=r[1]
10 PrevAsync 1 0 0 0
11 PrevAwait 1 0 0 0
12 Halt 0 0 0 0
13 Transaction 0 0 0 0 write=false
14 Goto 0 1 0 0
```
rowid seek, backwards:
```sql
limbo> explain select * from users where id < 100 order by id desc;
addr opcode p1 p2 p3 p4 p5 comment
---- ----------------- ---- ---- ---- ------------- -- -------
0 Init 0 19 0 0 Start at 19
1 OpenReadAsync 0 2 0 0 table=users, root=2
2 OpenReadAwait 0 0 0 0
3 Integer 100 11 0 0 r[11]=100
4 SeekLT 0 18 11 0
5 RowId 0 1 0 0 r[1]=users.rowid
6 Column 0 1 2 0 r[2]=users.first_name
7 Column 0 2 3 0 r[3]=users.last_name
8 Column 0 3 4 0 r[4]=users.email
9 Column 0 4 5 0 r[5]=users.phone_number
10 Column 0 5 6 0 r[6]=users.address
11 Column 0 6 7 0 r[7]=users.city
12 Column 0 7 8 0 r[8]=users.state
13 Column 0 8 9 0 r[9]=users.zipcode
14 Column 0 9 10 0 r[10]=users.age
15 ResultRow 1 10 0 0 output=r[1..10]
16 PrevAsync 0 0 0 0
17 PrevAwait 0 0 0 0
18 Halt 0 0 0 0
19 Transaction 0 0 0 0 write=false
20 Goto 0 1 0 0
```
two column order by, setup:
```sql
cargo run dualpk.db
Limbo v0.0.18-pre.3
Enter ".help" for usage hints.
limbo> .schema
CREATE TABLE a(b,c,d,e, primary key (d,c));
```
two column order by, forwards:
```sql
limbo> explain select * from a order by d,c;
addr opcode p1 p2 p3 p4 p5 comment
---- ----------------- ---- ---- ---- ------------- -- -------
0 Init 0 16 0 0 Start at 16
1 OpenReadAsync 0 2 0 0 table=a, root=2
2 OpenReadAwait 0 0 0 0
3 OpenReadAsync 1 3 0 0 table=sqlite_autoindex_a_1, root=3
4 OpenReadAwait 0 0 0 0
5 RewindAsync 1 0 0 0
6 RewindAwait 1 15 0 0 Rewind table sqlite_autoindex_a_1
7 DeferredSeek 1 0 0 0
8 Column 0 0 1 0 r[1]=a.b
9 Column 0 1 2 0 r[2]=a.c
10 Column 0 2 3 0 r[3]=a.d
11 Column 0 3 4 0 r[4]=a.e
12 ResultRow 1 4 0 0 output=r[1..4]
13 NextAsync 1 0 0 0
14 NextAwait 1 7 0 0
15 Halt 0 0 0 0
16 Transaction 0 0 0 0 write=false
17 Goto 0 1 0 0
```
two column order by, forwards with index seek:
```sql
limbo> explain select * from a where d > 100 order by d,c;
addr opcode p1 p2 p3 p4 p5 comment
---- ----------------- ---- ---- ---- ------------- -- -------
0 Init 0 16 0 0 Start at 16
1 OpenReadAsync 0 2 0 0 table=a, root=2
2 OpenReadAwait 0 0 0 0
3 OpenReadAsync 1 3 0 0 table=sqlite_autoindex_a_1, root=3
4 OpenReadAwait 0 0 0 0
5 Integer 100 5 0 0 r[5]=100
6 SeekGT 1 15 5 0
7 DeferredSeek 1 0 0 0
8 Column 0 0 1 0 r[1]=a.b
9 Column 0 1 2 0 r[2]=a.c
10 Column 0 2 3 0 r[3]=a.d
11 Column 0 3 4 0 r[4]=a.e
12 ResultRow 1 4 0 0 output=r[1..4]
13 NextAsync 1 0 0 0
14 NextAwait 1 7 0 0
15 Halt 0 0 0 0
16 Transaction 0 0 0 0 write=false
17 Goto 0 1 0 0
```
two column order by, forwards with index scan and termination condition:
```sql
limbo> explain select * from a where d < 100 order by d,c;
addr opcode p1 p2 p3 p4 p5 comment
---- ----------------- ---- ---- ---- ------------- -- -------
0 Init 0 18 0 0 Start at 18
1 OpenReadAsync 0 2 0 0 table=a, root=2
2 OpenReadAwait 0 0 0 0
3 OpenReadAsync 1 3 0 0 table=sqlite_autoindex_a_1, root=3
4 OpenReadAwait 0 0 0 0
5 Null 0 5 0 0 r[5]=NULL
6 SeekGT 1 17 5 0
7 Integer 100 6 0 0 r[6]=100
8 IdxGE 1 17 6 0
9 DeferredSeek 1 0 0 0
10 Column 0 0 1 0 r[1]=a.b
11 Column 0 1 2 0 r[2]=a.c
12 Column 0 2 3 0 r[3]=a.d
13 Column 0 3 4 0 r[4]=a.e
14 ResultRow 1 4 0 0 output=r[1..4]
15 NextAsync 1 0 0 0
16 NextAwait 1 7 0 0
17 Halt 0 0 0 0
18 Transaction 0 0 0 0 write=false
19 Goto 0 1 0 0
```
two column order by, backwards:
```sql
limbo> explain select * from a order by d desc,c desc;
addr opcode p1 p2 p3 p4 p5 comment
---- ----------------- ---- ---- ---- ------------- -- -------
0 Init 0 16 0 0 Start at 16
1 OpenReadAsync 0 2 0 0 table=a, root=2
2 OpenReadAwait 0 0 0 0
3 OpenReadAsync 1 3 0 0 table=sqlite_autoindex_a_1, root=3
4 OpenReadAwait 0 0 0 0
5 LastAsync 1 0 0 0
6 LastAwait 1 0 0 0
7 DeferredSeek 1 0 0 0
8 Column 0 0 1 0 r[1]=a.b
9 Column 0 1 2 0 r[2]=a.c
10 Column 0 2 3 0 r[3]=a.d
11 Column 0 3 4 0 r[4]=a.e
12 ResultRow 1 4 0 0 output=r[1..4]
13 PrevAsync 1 0 0 0
14 PrevAwait 1 0 0 0
15 Halt 0 0 0 0
16 Transaction 0 0 0 0 write=false
17 Goto 0 1 0 0
```
two column order by, backwards with index seek:
```sql
limbo> explain select * from a where d < 100 order by d desc,c desc;
addr opcode p1 p2 p3 p4 p5 comment
---- ----------------- ---- ---- ---- ------------- -- -------
0 Init 0 16 0 0 Start at 16
1 OpenReadAsync 0 2 0 0 table=a, root=2
2 OpenReadAwait 0 0 0 0
3 OpenReadAsync 1 3 0 0 table=sqlite_autoindex_a_1, root=3
4 OpenReadAwait 0 0 0 0
5 Integer 100 5 0 0 r[5]=100
6 SeekLT 1 15 5 0
7 DeferredSeek 1 0 0 0
8 Column 0 0 1 0 r[1]=a.b
9 Column 0 1 2 0 r[2]=a.c
10 Column 0 2 3 0 r[3]=a.d
11 Column 0 3 4 0 r[4]=a.e
12 ResultRow 1 4 0 0 output=r[1..4]
13 PrevAsync 1 0 0 0
14 PrevAwait 1 0 0 0
15 Halt 0 0 0 0
16 Transaction 0 0 0 0 write=false
17 Goto 0 1 0 0
```
two column order by, backwards with index scan and termination
condition:
```sql
limbo> explain select * from a where d > 100 order by d desc,c desc;
addr opcode p1 p2 p3 p4 p5 comment
---- ----------------- ---- ---- ---- ------------- -- -------
0 Init 0 18 0 0 Start at 18
1 OpenReadAsync 0 2 0 0 table=a, root=2
2 OpenReadAwait 0 0 0 0
3 OpenReadAsync 1 3 0 0 table=sqlite_autoindex_a_1, root=3
4 OpenReadAwait 0 0 0 0
5 LastAsync 1 0 0 0
6 LastAwait 1 0 0 0
7 Integer 100 6 0 0 r[6]=100
8 IdxLE 1 17 6 0
9 DeferredSeek 1 0 0 0
10 Column 0 0 1 0 r[1]=a.b
11 Column 0 1 2 0 r[2]=a.c
12 Column 0 2 3 0 r[3]=a.d
13 Column 0 3 4 0 r[4]=a.e
14 ResultRow 1 4 0 0 output=r[1..4]
15 PrevAsync 1 0 0 0
16 PrevAwait 1 0 0 0
17 Halt 0 0 0 0
18 Transaction 0 0 0 0 write=false
19 Goto 0 1 0 0
```
Reviewed-by: Preston Thorpe (@PThorpe92)
Closes #1209
2025-04-09 12:03:14 +03:00
Pere Diaz Bou
edc3a420fb
comment how page count is decreased while balancing
2025-04-09 11:02:49 +02:00
Jussi Saurio
0888c71ba0
use seek() instead of do_seek() to set iteration state
2025-04-09 10:26:02 +03:00
Jussi Saurio
7435d005db
Merge ' allow insertion of multiple overflow cells ' from Pere Diaz Bou
...
With big cells it is easy to see multiple overflow cells happen while
balancing. Previously we disallowed insertion with > 1 overflow cells,
let's fix this by allowing it as we should be safe now.
This pr also adds some more comments to the balancing algorithm.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com >
Closes #1270
2025-04-09 10:17:36 +03:00
Jussi Saurio
5e3a37a192
Try to name iteration direction sensitive method better
2025-04-09 10:14:29 +03:00
Jussi Saurio
3124fca5b7
Dereference instead of explicit clone
2025-04-09 10:14:29 +03:00
Jussi Saurio
0bb87b060a
Fix existing table btree backwards iteration logic
2025-04-09 10:14:29 +03:00
Jussi Saurio
d9bae633c0
Add rowid_seek_fuzz() test
2025-04-09 10:14:29 +03:00
Jussi Saurio
f5220d281d
Fix off-by-one logic in btree table traversal
2025-04-09 10:14:29 +03:00
Jussi Saurio
fa295af635
Fix insert fuzz test by bypassing internal invariant
2025-04-09 10:14:29 +03:00
Jussi Saurio
024c63f808
optimizer: remove ORDER BY if index can be used to satisfy the order
2025-04-09 10:14:29 +03:00
Jussi Saurio
a706b7160a
planner: support index backwards seeks and iteration
2025-04-09 10:14:29 +03:00
Jussi Saurio
c9190236f0
btree: support backwards index seeks and iteration
2025-04-09 10:14:29 +03:00
Jussi Saurio
3e42a62cd0
Add SeekLE/SeekLT operations to VDBE
2025-04-09 10:14:29 +03:00
Jussi Saurio
431ef2fa6a
Add TCL/differential fuzz tests for verifying index scan behavior
2025-04-09 10:14:29 +03:00
Jussi Saurio
ed929d2a0a
Merge 'Properly handle insertion of indexed columns' from Preston Thorpe
...
```console
limbo> create index p on products(price);
limbo> explain insert into products (name,price) values ('huh',32);
addr opcode p1 p2 p3 p4 p5 comment
---- ----------------- ---- ---- ---- ------------- -- -------
0 Init 0 17 0 0 Start at 17
1 OpenWriteAsync 0 273 0 0
2 OpenWriteAwait 0 0 0 0
3 String8 0 3 0 huh 0 r[3]='huh'
4 Integer 32 4 0 0 r[4]=32
5 OpenWriteAsync 1 297 0 0
6 OpenWriteAwait 0 0 0 0
7 NewRowId 0 1 0 0
8 MakeRecord 2 3 5 0 r[5]=mkrec(r[2..4])
9 InsertAsync 0 5 1 0
10 InsertAwait 0 0 0 0
11 Copy 4 6 0 0 r[6]=r[4]
12 Copy 1 7 0 0 r[7]=r[1]
13 MakeRecord 6 2 8 0 r[8]=mkrec(r[6..7])
14 IdxInsertAsync 1 8 6 0 key=r[8]
15 IdxInsertAwait 1 0 0 0
16 Halt 0 0 0 0
17 Transaction 0 1 0 0 write=true
18 Null 0 2 0 0 r[2]=NULL
19 Goto 0 1 0 0
```
When an insert happens, we need to be sure to insert into the index
btree as well.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com >
Closes #1265
2025-04-09 10:13:59 +03:00
Jussi Saurio
08b352cd81
Merge 'VTabs: Proper handling of re-opened db files without the relevant extensions loaded' from Preston Thorpe
...
closes #1239
warns the user if a .db file is opened that contains a schema entry of a
vtab created with a module that is no longer loaded.
If the module is loaded for that connection, the user can properly query
the table.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com >
Closes #1244
2025-04-09 09:36:26 +03:00
PThorpe92
9b1e60a29c
Fix typo in ext library lock err message
2025-04-08 20:10:50 -04:00
PThorpe92
41ac91f14f
Add tests for parsing vtab creation sql in ParseSchema
2025-04-08 20:10:49 -04:00
PThorpe92
3a7f1e4056
Add comments explaining flow of reloading vtabs from schema tbl
2025-04-08 20:10:49 -04:00
PThorpe92
6b5ec1f07b
Remove mut borrow from sym table in parse schema fn
2025-04-08 20:10:49 -04:00
