Jussi Saurio
d286a56e15
refactor: fold Async/Await insns into a single instruction
2025-04-14 09:40:20 +03:00
Diego Reis
fd79ad2644
core/vdbe: Change is_btree to is_table in OpenEphemeral
2025-04-13 11:15:01 -03:00
Diego Reis
4c315e1bb6
core/vdbe: Update OpenEphemeral to use CreateBtreeFlags
2025-04-13 11:13:25 -03:00
Diego Reis
035e6dcef4
core/vdbe: Fix logic error during btree creation
...
I do thing we should change this 1,2 flag to 0,1 or just an enum, to be more rustacean. The current state can be very misleading
2025-04-13 11:10:06 -03:00
Diego Reis
61c324cca5
core/vdbe: Add missing work to get cursor and transient table usable
2025-04-13 11:10:06 -03:00
Diego Reis
bcac1fe778
core/vdbe: Rename page_io to db_file in OpenEphemeral
2025-04-13 11:10:06 -03:00
Diego Reis
d9bf383507
core/io: Untie MemoryIO's lifetime of the IO layer
2025-04-13 11:10:06 -03:00
Diego Reis
79f8b83cbe
Fix dumb clippy errors
2025-04-13 11:10:06 -03:00
Diego Reis
66e12e1c2d
core/vdbe: Create OpenEphemeral bytecode
...
"Open a new cursor P1 to a transient table. The cursor is always opened read/write even if the main database is read-only. The ephemeral table is deleted automatically when the cursor is closed.
If the cursor P1 is already opened on an ephemeral table, the table is cleared (all content is erased)."
There is still some work to do, but this is a basic setup
2025-04-13 11:10:06 -03:00
Jussi Saurio
8e601959ad
Merge 'Fuzz fix some operations' from Pedro Muniz
...
Me and @diegoreis42 were late at night, running the new fuzzer and saw
some errors with `Bit-Not` and `Not` operations. Added some tests as
well for those fixes.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com >
Closes #1328
2025-04-13 17:02:19 +03:00
Jussi Saurio
cbe3500b7e
Merge 'Code clean-ups' from Diego Reis
...
While developing I found that some things could be improved :)
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com >
Closes #1325
2025-04-13 17:02:12 +03:00
pedrocarlo
1297cb107c
bit-not and boolean-not
...
Co-authored-by: Diego Reis <79876389+diegoreis42@users.noreply.github.com >
2025-04-13 02:45:12 -03:00
Diego Reis
51eb2af06a
core(refactor): Add CreateBTreeFlags
...
Passing 1s and 0s with comments is not rustacean, and since we already follow the pattern of struct flags in other sections of the codebase it's better use it here too.
2025-04-13 01:46:30 -03:00
Levy A.
d210ee1497
cargo fmt
2025-04-12 18:55:48 -03:00
Levy A.
5c0b112125
fix: return null when parameter is unbound
2025-04-12 17:43:04 -03:00
Pekka Enberg
e3a4400329
Merge 'Multi column indexes + index seek refactor' from Jussi Saurio
...
# Multi column indexes + index seek refactor
## PR reader guide
I would say mostly you should just focus on the content of
`optimizer.rs` and `plan.rs` because the rest is just small type
changes, or in the case of `main_loop.rs`, a bunch of logic was just
moved out of there and rewritten.
## New feature - multi column index seeks
This PR adds support for utilizing multi-column indexes properly, i.e.
using as many columns in the seek key as possible. Previously, we only
used max one column per index. I've modified the existing compound index
seek fuzz test to use this functionality.
## Refactoring of index seek related logic
This PR moves a lot of index seek related logic out of `main_loop.rs`
into `optimizer.rs` and `plan.rs` and introduces a bunch of helper
structures to model finding and using an index to perform a seek + scan.
## Examples
Here are some examples of multi-column seeks:
### Example table setup:
```sql
sqlite> CREATE TABLE t(a,b,c,d,e);
sqlite> CREATE INDEX abc ON t (a,b,c);
-- create 10000 rows with random values between 0-9 for all columns
sqlite >INSERT INTO t SELECT ABS(RANDOM() % 10),ABS(RANDOM() % 10),ABS(RANDOM() % 10),ABS(RANDOM() % 10),ABS(RANDOM() % 10) FROM generate_series(1,10000,1);
```
### Example bytecode plans, results and timings vs main branch:
```sql
limbo> EXPLAIN SELECT * FROM t WHERE a = 5 and b = 6 and c = 7;
addr opcode p1 p2 p3 p4 p5 comment
---- ----------------- ---- ---- ---- ------------- -- -------
0 Init 0 20 0 0 Start at 20
1 OpenReadAsync 0 2 0 0 table=t, root=2
2 OpenReadAwait 0 0 0 0
3 OpenReadAsync 1 3 0 0 table=abc, root=3
4 OpenReadAwait 0 0 0 0
5 Integer 5 6 0 0 r[6]=5
6 Integer 6 7 0 0 r[7]=6
7 Integer 7 8 0 0 r[8]=7
8 SeekGE 1 19 6 0 key=[6..8]
9 IdxGT 1 19 6 0 key=[6..8]
10 DeferredSeek 1 0 0 0
11 Column 0 0 1 0 r[1]=t.a
12 Column 0 1 2 0 r[2]=t.b
13 Column 0 2 3 0 r[3]=t.c
14 Column 0 3 4 0 r[4]=t.d
15 Column 0 4 5 0 r[5]=t.e
16 ResultRow 1 5 0 0 output=r[1..5]
17 NextAsync 1 0 0 0
18 NextAwait 1 9 0 0
19 Halt 0 0 0 0
20 Transaction 0 0 0 0 write=false
21 Goto 0 1 0 0
limbo> SELECT * FROM t WHERE a = 5 and b = 6 and c = 7;
5|6|7|9|9
5|6|7|4|7
5|6|7|3|2
5|6|7|3|7
5|6|7|5|2
5|6|7|5|3
5|6|7|9|7
runtime (debug build, this branch): total: 2 ms (this includes parsing/coloring of cli app)
runtime (debud build, main branch): total: 67 ms (this includes parsing/coloring of cli app)
```
```sql
limbo> EXPLAIN SELECT * FROM t WHERE a = 5 and b = 6 and c < 7;
addr opcode p1 p2 p3 p4 p5 comment
---- ----------------- ---- ---- ---- ------------- -- -------
0 Init 0 21 0 0 Start at 21
1 OpenReadAsync 0 2 0 0 table=t, root=2
2 OpenReadAwait 0 0 0 0
3 OpenReadAsync 1 3 0 0 table=abc, root=3
4 OpenReadAwait 0 0 0 0
5 Integer 5 6 0 0 r[6]=5
6 Integer 6 7 0 0 r[7]=6
7 Null 0 8 0 0 r[8]=NULL
8 SeekGT 1 20 6 0 key=[6..8]
9 Integer 7 8 0 0 r[8]=7
10 IdxGE 1 20 6 0 key=[6..8]
11 DeferredSeek 1 0 0 0
12 Column 0 0 1 0 r[1]=t.a
13 Column 0 1 2 0 r[2]=t.b
14 Column 0 2 3 0 r[3]=t.c
15 Column 0 3 4 0 r[4]=t.d
16 Column 0 4 5 0 r[5]=t.e
17 ResultRow 1 5 0 0 output=r[1..5]
18 NextAsync 1 0 0 0
19 NextAwait 1 10 0 0
20 Halt 0 0 0 0
21 Transaction 0 0 0 0 write=false
22 Goto 0 1 0 0
limbo> SELECT * FROM t WHERE a = 5 and b = 6 and c < 7;
5|6|0|0|3
5|6|0|5|1
5|6|0|3|1
5|6|0|6|3
5|6|0|8|1
5|6|0|2|7
5|6|0|9|9
5|6|0|5|3
5|6|0|4|2
5|6|0|4|2
5|6|0|0|2
5|6|0|7|2
5|6|1|8|5
5|6|1|7|5
5|6|1|7|2
5|6|1|1|2
5|6|1|6|5
5|6|1|1|5
5|6|1|5|7
5|6|1|1|9
5|6|1|4|3
5|6|1|1|2
5|6|1|2|2
5|6|1|4|4
5|6|1|9|6
5|6|1|2|5
5|6|1|2|4
5|6|1|7|1
5|6|2|0|9
5|6|2|6|9
5|6|2|4|5
5|6|2|9|3
5|6|2|5|2
5|6|2|9|0
5|6|2|7|1
5|6|3|6|5
5|6|3|8|5
5|6|3|5|4
5|6|3|5|2
5|6|3|1|1
5|6|3|2|0
5|6|3|9|3
5|6|3|6|9
5|6|3|7|6
5|6|3|3|5
5|6|3|0|8
5|6|3|6|4
5|6|4|1|1
5|6|4|9|8
5|6|4|3|7
5|6|4|1|3
5|6|4|8|9
5|6|4|9|7
5|6|4|7|9
5|6|4|8|8
5|6|4|3|1
5|6|4|2|6
5|6|4|5|7
5|6|4|2|6
5|6|4|4|3
5|6|5|2|4
5|6|5|6|7
5|6|5|3|8
5|6|5|7|8
5|6|5|9|6
5|6|5|2|7
5|6|5|1|7
5|6|5|0|6
5|6|6|2|4
5|6|6|9|4
5|6|6|4|9
5|6|6|5|6
5|6|6|2|2
5|6|6|0|6
runtime (debug build, this branch): total: 9 ms (this includes parsing/coloring of cli app)
runtime (debug build, main branch): total: 71 ms (this includes parsing/coloring of cli app)
```
```sql
limbo> EXPLAIN SELECT * FROM t WHERE a = 5 and b = 6 and c < 7 ORDER BY a desc, b desc, c desc;
addr opcode p1 p2 p3 p4 p5 comment
---- ----------------- ---- ---- ---- ------------- -- -------
0 Init 0 20 0 0 Start at 20
1 OpenReadAsync 0 2 0 0 table=t, root=2
2 OpenReadAwait 0 0 0 0
3 OpenReadAsync 1 3 0 0 table=abc, root=3
4 OpenReadAwait 0 0 0 0
5 Integer 5 6 0 0 r[6]=5
6 Integer 6 7 0 0 r[7]=6
7 Integer 7 8 0 0 r[8]=7
8 SeekLT 1 19 6 0 key=[6..8]
9 IdxLT 1 19 6 0 key=[6..7]
10 DeferredSeek 1 0 0 0
11 Column 0 0 1 0 r[1]=t.a
12 Column 0 1 2 0 r[2]=t.b
13 Column 0 2 3 0 r[3]=t.c
14 Column 0 3 4 0 r[4]=t.d
15 Column 0 4 5 0 r[5]=t.e
16 ResultRow 1 5 0 0 output=r[1..5]
17 PrevAsync 1 0 0 0
18 PrevAwait 1 0 0 0
19 Halt 0 0 0 0
20 Transaction 0 0 0 0 write=false
21 Goto 0 1 0 0
limbo> SELECT * FROM t WHERE a = 5 and b = 6 and c < 7 ORDER BY a desc, b desc, c desc;
5|6|6|0|6
5|6|6|2|2
5|6|6|5|6
5|6|6|4|9
5|6|6|9|4
5|6|6|2|4
5|6|5|0|6
5|6|5|1|7
5|6|5|2|7
5|6|5|9|6
5|6|5|7|8
5|6|5|3|8
5|6|5|6|7
5|6|5|2|4
5|6|4|4|3
5|6|4|2|6
5|6|4|5|7
5|6|4|2|6
5|6|4|3|1
5|6|4|8|8
5|6|4|7|9
5|6|4|9|7
5|6|4|8|9
5|6|4|1|3
5|6|4|3|7
5|6|4|9|8
5|6|4|1|1
5|6|3|6|4
5|6|3|0|8
5|6|3|3|5
5|6|3|7|6
5|6|3|6|9
5|6|3|9|3
5|6|3|2|0
5|6|3|1|1
5|6|3|5|2
5|6|3|5|4
5|6|3|8|5
5|6|3|6|5
5|6|2|7|1
5|6|2|9|0
5|6|2|5|2
5|6|2|9|3
5|6|2|4|5
5|6|2|6|9
5|6|2|0|9
5|6|1|7|1
5|6|1|2|4
5|6|1|2|5
5|6|1|9|6
5|6|1|4|4
5|6|1|2|2
5|6|1|1|2
5|6|1|4|3
5|6|1|1|9
5|6|1|5|7
5|6|1|1|5
5|6|1|6|5
5|6|1|1|2
5|6|1|7|2
5|6|1|7|5
5|6|1|8|5
5|6|0|7|2
5|6|0|0|2
5|6|0|4|2
5|6|0|4|2
5|6|0|5|3
5|6|0|9|9
5|6|0|2|7
5|6|0|8|1
5|6|0|6|3
5|6|0|3|1
5|6|0|5|1
5|6|0|0|3
runtime (debug build, this branch): total: 9 ms (this includes parsing/coloring of cli app)
runtime (debug build, main branch): total: 71 ms (this includes parsing/coloring of cli app)
```
Closes #1288
2025-04-11 09:36:25 +03:00
Pekka Enberg
d67e1b604b
Merge 'Added 'likelihood' scalar function' from Sachin Kumar Singh
...
The `likelihood(X,Y)` function returns argument X unchanged. The value Y
in likelihood(X,Y) must be a floating point constant between 0.0 and
1.0, inclusive.
```
sqlite> explain SELECT likelihood(42, 0.0);
addr opcode p1 p2 p3 p4 p5 comment
---- ------------- ---- ---- ---- ------------- -- -------------
0 Init 0 6 0 0 Start at 6
1 Once 0 3 0 0
2 Integer 42 2 0 0 r[2]=42
3 Copy 2 1 0 0 r[1]=r[2]
4 ResultRow 1 1 0 0 output=r[1]
5 Halt 0 0 0 0
6 Goto 0 1 0 0
```
```
limbo> explain SELECT likelihood(42, 0.0);
addr opcode p1 p2 p3 p4 p5 comment
---- ----------------- ---- ---- ---- ------------- -- -------
0 Init 0 4 0 0 Start at 4
1 Copy 2 1 0 0 r[1]=r[2]
2 ResultRow 1 1 0 0 output=r[1]
3 Halt 0 0 0 0
4 Integer 42 2 0 0 r[2]=42
5 Goto 0 1 0 0
```
Closes #1303
2025-04-11 09:34:36 +03:00
Sachin Singh
01fa02364d
correctly handle edge cases
2025-04-11 08:34:29 +05:30
Sachin Singh
482e93bfd0
feat: add likelihood scalar function
2025-04-11 05:54:23 +05:30
Sachin Singh
b7acfa490c
feat: add timediff data and time function
2025-04-11 04:30:57 +05:30
Pere Diaz Bou
8e93471d00
fix cell index selection while balancing
...
Cell index doesn't move in `move_to` unless we don't need to check next
cell. On the other hand, with rightmost pointer, we advance cell index
by 1 even though where we are moving to was to that page
2025-04-10 16:01:24 +02:00
Jussi Saurio
afad06fb23
vdbe/explain: add key info to Seek/Idx insns
2025-04-10 15:06:45 +03:00
Pekka Enberg
31f0d174d7
core/vdbe: Move exec_*() funtions to execute.rs
2025-04-10 09:42:03 +03:00
Pekka Enberg
3fd51cdf06
core/vdbe: Move Insn implementation close to struct definition
2025-04-10 09:28:43 +03:00
Pekka Enberg
5906d7971a
core/vdbe: Clean up imports
2025-04-10 09:25:15 +03:00
PThorpe92
13ae19c78c
Remove unnecessary clones from mc cursors
2025-04-09 11:15:04 -04:00
PThorpe92
62d1447cd6
Adapt query plan to handle vatbs for updates
2025-04-09 11:15:02 -04:00
Pekka Enberg
ddc5e49451
Merge 'Index insert fixes' from Pere Diaz Bou
...
Closes #1279
2025-04-09 17:21:53 +03:00
Pekka Enberg
a4d9f70ef8
Merge 'Strict table support' from Ihor Andrianov
...
Closes #884
Support for
```CREATE TABLE test(id INTEGER) STRICT;```
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com >
Closes #1268
2025-04-09 16:45:56 +03:00
Pere Diaz Bou
12899034c9
make insert idx re-entrant
2025-04-09 15:04:45 +02:00
Jussi Saurio
3e42a62cd0
Add SeekLE/SeekLT operations to VDBE
2025-04-09 10:14:29 +03:00
Jussi Saurio
ed929d2a0a
Merge 'Properly handle insertion of indexed columns' from Preston Thorpe
...
```console
limbo> create index p on products(price);
limbo> explain insert into products (name,price) values ('huh',32);
addr opcode p1 p2 p3 p4 p5 comment
---- ----------------- ---- ---- ---- ------------- -- -------
0 Init 0 17 0 0 Start at 17
1 OpenWriteAsync 0 273 0 0
2 OpenWriteAwait 0 0 0 0
3 String8 0 3 0 huh 0 r[3]='huh'
4 Integer 32 4 0 0 r[4]=32
5 OpenWriteAsync 1 297 0 0
6 OpenWriteAwait 0 0 0 0
7 NewRowId 0 1 0 0
8 MakeRecord 2 3 5 0 r[5]=mkrec(r[2..4])
9 InsertAsync 0 5 1 0
10 InsertAwait 0 0 0 0
11 Copy 4 6 0 0 r[6]=r[4]
12 Copy 1 7 0 0 r[7]=r[1]
13 MakeRecord 6 2 8 0 r[8]=mkrec(r[6..7])
14 IdxInsertAsync 1 8 6 0 key=r[8]
15 IdxInsertAwait 1 0 0 0
16 Halt 0 0 0 0
17 Transaction 0 1 0 0 write=true
18 Null 0 2 0 0 r[2]=NULL
19 Goto 0 1 0 0
```
When an insert happens, we need to be sure to insert into the index
btree as well.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com >
Closes #1265
2025-04-09 10:13:59 +03:00
PThorpe92
6b5ec1f07b
Remove mut borrow from sym table in parse schema fn
2025-04-08 20:10:49 -04:00
PThorpe92
c15035caf8
Add module and vtab to schema after table is reopened with proper ext
2025-04-08 20:10:48 -04:00
PThorpe92
4b9b6c969b
Parse schema rows after extensions are loaded
2025-04-08 20:10:47 -04:00
PThorpe92
1f29307fe8
Support proper index handling when doing insertions
2025-04-08 08:55:14 -04:00
Ihor Andrianov
7c15465118
add TypeCheck insn to update
2025-04-07 20:02:14 +03:00
Ihor Andrianov
4a08b98bab
implemented strict table
2025-04-07 20:01:39 +03:00
jachewz
db15661b7e
fix: i64 % -1 overflow panic
2025-04-07 22:03:48 +10:00
jachewz
a72b75e193
fix: remainder operator rhs text
2025-04-07 22:03:48 +10:00
Duncan Lutz
aa7c64cb19
feat: added likely scalar function
2025-04-06 23:14:30 -06:00
Pekka Enberg
2d3fd01f91
Merge 'Support Create Index' from Preston Thorpe
...
Closes #1193
```console
│limbo> explain create index idxp on products(price);
│addr opcode p1 p2 p3 p4 p5 comment
│---- ----------------- ---- ---- ---- ------------- -- -------
│0 Init 0 39 0 0 Start at 39
│1 CreateBtree 0 1 2 0 r[1]=root iDb=0 flags=2
│2 OpenWriteAsync 0 1 0 0
│3 OpenWriteAwait 0 0 0 0
│4 NewRowId 0 2 0 0
│5 String8 0 3 0 index 0 r[3]='index'
│6 String8 0 4 0 idxp 0 r[4]='idxp'
│7 String8 0 5 0 products 0 r[5]='products'
│8 Copy 1 6 1 0 r[6]=r[1]
│9 String8 0 7 0 CREATE INDEX idxp ON products (price) 0 r[7]='CREATE INDEX idxp ON products (price)'
│10 MakeRecord 3 5 8 0 r[8]=mkrec(r[3..7])
│11 InsertAsync 0 8 2 0
│12 InsertAwait 0 0 0 0
│13 SorterOpen 3 1 0 k(1,B) 0 cursor=3
│14 OpenPseudo 4 9 2 0 2 columns in r[9]
│15 OpenReadAsync 2 273 0 0 table=products, root=273
│16 OpenReadAwait 0 0 0 0
│17 RewindAsync 2 0 0 0
│18 RewindAwait 2 25 0 0 Rewind table products
│19 Column 2 2 10 0 r[10]=products.price
│20 RowId 2 11 0 0 r[11]=products.rowid
│21 MakeRecord 10 2 12 0 r[12]=mkrec(r[10..11])
│22 SorterInsert 3 12 0 0 0 key=r[12]
│23 NextAsync 2 0 0 0
│24 NextAwait 2 19 0 0
│25 OpenWriteAsync 1 1 0 0
│26 OpenWriteAwait 0 0 0 0
│27 SorterSort 3 33 0 0
│28 SorterData 3 13 4 0 r[13]=data
│29 SeekEnd 1 0 0 0
│30 IdxInsertAsync 1 13 0 0 key=r[13]
│31 IdxInsertAwait 1 0 0 0
│32 SorterNext 3 28 0 0
│33 Close 3 0 0 0
│34 Close 2 0 0 0
│35 Close 1 0 0 0
│36 ParseSchema 0 0 0 name = 'idxp' AND type = 'index' 0 name = 'idxp' AND type = 'index'
│37 Close 0 0 0 0
│38 Halt 0 0 0 0
│39 Transaction 0 1 0 0 write=true
│40 Goto 0 1 0 0
```
This will create the initial index btree and insert whatever relevant
records that need to be inserted, it doesn't handle the case of
inserting new index keys when normal records are created afterwards.
That will prob be added in next PR to keep this one concise.
Limbo will properly use the index in a subsequent query:
Creating a unique index on a column that has 2 existing identical rows:
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com >
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com >
Closes #1199
2025-04-06 19:32:36 +03:00
PThorpe92
67eda10453
Allow reading altered tables by defaulting to null in Column insn
2025-04-05 16:19:56 -04:00
PThorpe92
83af71f140
Return accidentally deleted comment on SeekGE insn from merge conflict
2025-04-05 11:30:57 -04:00
PThorpe92
a2b9d8d371
Use Correct flag on idx insert to prevent seeking
2025-04-05 11:19:09 -04:00
PThorpe92
068ab4ab27
Refactor btree to reuse existing insert and seek with idx keys
2025-04-05 11:19:09 -04:00
PThorpe92
e020ba3dfe
Add enum for interpreting a value as a register or literal for insns
2025-04-05 11:19:07 -04:00
PThorpe92
b0016a0ee2
Support create index with SeekEnd and IdxCreate opcode functionality
2025-04-05 11:15:36 -04:00
PThorpe92
45a8e5e226
Add close_cursors helper method to program builder
2025-04-05 11:06:18 -04:00
Pekka Enberg
c3eaf47180
Merge 'Add support for default values in INSERT statements' from Diego Reis
...
While working on #494 I noticed that default values defined in schemas
weren't being applied.
Before:
Now:
Closes #1249
2025-04-04 08:59:44 +03:00
