aljaz/turso
1
0
Fork 0
mirror of https://github.com/aljazceru/turso.git synced 2026-01-05 09:14:24 +01:00
Code Issues Packages Projects Releases Wiki Activity
1,002 Commits 1 Branch 0 Tags
fdf3b0d16bcdc9bed7b46ddf41aa8441558df8ae
Commit Graph

601 Commits

Author SHA1 Message Date
Pekka Enberg
fdf3b0d16b Merge '(core): Primary key index scans and single-column secondary index scans' from Jussi Saurio 
This PR adds an index on `users.age` to `testing.db`, and support for
indexed lookups. Only single-column ascending indexes are currently
supported.
This PR also gets rid of `Operator::Seekrowid` in favor of
`Operator::Search` which handles all non-full-table-scan searches: 1.
integer primary key (rowid) point queries 2. integer primary key index
scans, and 3. secondary index scans.
examples:
```
limbo> select first_name, age from users where age > 90 limit 10;
Miranda|90
Sarah|90
Justin|90
Justin|90
John|90
Jeremy|90
Stephanie|90
Joshua|90
Jenny|90
Jennifer|90

limbo> explain query plan select first_name, age from users where age > 90 limit 10;
QUERY PLAN
`--TAKE 10
   `--PROJECT first_name, age
   |  `--SEARCH users USING INDEX age_idx

limbo> explain select first_name, age from users where age > 90 limit 10;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     15    0                    0   Start at 15
1     OpenReadAsync      0     2     0                    0   table=users, root=2
2     OpenReadAwait      0     0     0                    0
3     OpenReadAsync      1     274   0                    0   table=age_idx, root=274
4     OpenReadAwait      0     0     0                    0
5     Integer            90    1     0                    0   r[1]=90
6     SeekGT             1     14    1                    0
7       DeferredSeek     1     0     0                    0
8       Column           0     1     2                    0   r[2]=users.first_name
9       Column           0     9     3                    0   r[3]=users.age
10      ResultRow        2     2     0                    0   output=r[2..3]
11      DecrJumpZero     4     14    0                    0   if (--r[4]==0) goto 14
12    NextAsync          1     0     0                    0
13    NextAwait          1     7     0                    0
14    Halt               0     0     0                    0
15    Transaction        0     0     0                    0
16    Integer            10    4     0                    0   r[4]=10
17    Goto               0     1     0                    0
```
Sqlite version:
```
sqlite> explain select first_name, age from users where age > 90 limit 10;
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     13    0                    0   Start at 13
1     Integer        10    1     0                    0   r[1]=10; LIMIT counter
2     OpenRead       0     2     0     10             0   root=2 iDb=0; users
3     OpenRead       1     274   0     k(2,,)         0   root=274 iDb=0; age_idx
4     Integer        90    2     0                    0   r[2]=90
5     SeekGT         1     12    2     1              0   key=r[2]
6       DeferredSeek   1     0     0                    0   Move 0 to 1.rowid if needed
7       Column         0     1     3                    0   r[3]= cursor 0 column 1
8       Column         1     0     4                    0   r[4]= cursor 1 column 0
9       ResultRow      3     2     0                    0   output=r[3..4]
10      DecrJumpZero   1     12    0                    0   if (--r[1])==0 goto 12
11    Next           1     6     0                    0
12    Halt           0     0     0                    0
13    Transaction    0     0     3     0              1   usesStmtJournal=0
14    Goto           0     1     0                    0
```
---
´Seek` instructions are also now supported for primary key rowid
searches:
```
limbo> select id, first_name from users where id > 9995;
9996|Donald
9997|Ruth
9998|Dorothy
9999|Gina
10000|Nicole

limbo> explain query plan select id, first_name from users where id > 9995;
QUERY PLAN
`--PROJECT id, first_name
   `--SEARCH users USING INTEGER PRIMARY KEY (rowid=?)

limbo> explain select id, first_name from users where id > 9995;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     11    0                    0   Start at 11
1     OpenReadAsync      0     2     0                    0   table=users, root=2
2     OpenReadAwait      0     0     0                    0
3     Integer            9995  1     0                    0   r[1]=9995
4     SeekGT             0     10    1                    0
5       RowId            0     2     0                    0   r[2]=users.rowid
6       Column           0     1     3                    0   r[3]=users.first_name
7       ResultRow        2     2     0                    0   output=r[2..3]
8     NextAsync          0     0     0                    0
9     NextAwait          0     5     0                    0
10    Halt               0     0     0                    0
11    Transaction        0     0     0                    0
12    Goto               0     1     0                    0
```
sqlite:
```
sqlite> explain select id, first_name from users where id > 9995;
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     8     0                    0   Start at 8
1     OpenRead       0     2     0     2              0   root=2 iDb=0; users
2     SeekGT         0     7     1                    0   key=r[1]; pk
3       Rowid          0     2     0                    0   r[2]=users.rowid
4       Column         0     1     3                    0   r[3]= cursor 0 column 1
5       ResultRow      2     2     0                    0   output=r[2..3]
6     Next           0     3     0                    0
7     Halt           0     0     0                    0
8     Transaction    0     0     3     0              1   usesStmtJournal=0
9     Integer        9995  1     0                    0   r[1]=9995
10    Goto           0     1     0                    0
```
---
More complex example with a join that uses both a rowid lookup and a
secondary index scan:
```
limbo> explain query plan select u.first_name, p.name from users u join products p on u.id = p.id and u.age > 70;
QUERY PLAN
`--PROJECT u.first_name, p.name
   `--JOIN
   |  |--SEARCH u USING INDEX age_idx
   |  `--SEARCH p USING INTEGER PRIMARY KEY (rowid=?)

limbo> explain select u.first_name, p.name from users u join products p on u.id = p.id and u.age > 70;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     18    0                    0   Start at 18
1     OpenReadAsync      0     2     0                    0   table=u, root=2
2     OpenReadAwait      0     0     0                    0
3     OpenReadAsync      1     274   0                    0   table=age_idx, root=274
4     OpenReadAwait      0     0     0                    0
5     OpenReadAsync      2     3     0                    0   table=p, root=3
6     OpenReadAwait      0     0     0                    0
7     Integer            70    1     0                    0   r[1]=70
8     SeekGT             1     17    1                    0
9       DeferredSeek     1     0     0                    0
10      RowId            0     2     0                    0   r[2]=u.rowid
11      SeekRowid        2     2     15                   0   if (r[2]!=p.rowid) goto 15
12      Column           0     1     3                    0   r[3]=u.first_name
13      Column           2     1     4                    0   r[4]=p.name
14      ResultRow        3     2     0                    0   output=r[3..4]
15    NextAsync          1     0     0                    0
16    NextAwait          1     9     0                    0
17    Halt               0     0     0                    0
18    Transaction        0     0     0                    0
19    Goto               0     1     0                    0
```
sqlite:
```
sqlite> explain select u.first_name, p.name from users u join products p on u.id = p.id and u.age > 70;
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     14    0                    0   Start at 14
1     OpenRead       0     2     0     10             0   root=2 iDb=0; users
2     OpenRead       2     274   0     k(2,,)         0   root=274 iDb=0; age_idx
3     OpenRead       1     3     0     2              0   root=3 iDb=0; products
4     Integer        70    1     0                    0   r[1]=70
5     SeekGT         2     13    1     1              0   key=r[1]
6       DeferredSeek   2     0     0                    0   Move 0 to 2.rowid if needed
7       IdxRowid       2     2     0                    0   r[2]=rowid; users.rowid
8       SeekRowid      1     12    2                    0   intkey=r[2]
9       Column         0     1     3                    0   r[3]= cursor 0 column 1
10      Column         1     1     4                    0   r[4]= cursor 1 column 1
11      ResultRow      3     2     0                    0   output=r[3..4]
12    Next           2     6     0                    0
13    Halt           0     0     0                    0
14    Transaction    0     0     3     0              1   usesStmtJournal=0
15    Goto           0     1     0                    0
```

Closes #350
 2024-10-13 10:11:23 +03:00
Arpit Saxena
28a603f56a [sorter] Hold records in Vec instead of a BTreeMap 
We now just insert them one after the other in the vector. When rewind
is called, the vector is sorted. Iterating is just taking elements from
the vector
 2024-10-09 17:58:28 +05:30
jussisaurio
556f4b73c9 Refine edge case handling: add optional predicate to get_next_record() 2024-10-08 08:23:30 +03:00
jussisaurio
43038cb6aa Handle seek() edge case with index seek 2024-10-08 07:45:21 +03:00
jussisaurio
572db69b5e Add TODO comment about index corner case 2024-10-07 17:12:19 +03:00
jussisaurio
93a8110773 dont assume index key has rowid in the second column: its the last 2024-10-07 17:05:38 +03:00
jussisaurio
8563d620af renaming 2024-10-07 17:03:50 +03:00
jussisaurio
fc71f2b32f traverse index properly 
interior index cells have values that are not in the leaves, e.g.

     (interior: 3)
    /            \
(leaf: 2)     (leaf: 4)

so their values need to be emitted after the left subtree is emitted.
 2024-10-07 13:04:03 +03:00
jussisaurio
e5cf052f07 Why do sqlite btree child keys have <= keys and not < keys 2024-10-06 23:48:59 +03:00
Lauri Virtanen
0ae1412193 Add instr(X,Y) scalar function 
Relates to issue #144
 2024-10-06 20:19:37 +03:00
jussisaurio
15a66ea662 single seek function in cursor trait 2024-10-06 09:21:15 +03:00
jussisaurio
6e7db36121 reorder 2024-10-06 00:58:32 +03:00
jussisaurio
af9a751d36 Single seek function 2024-10-06 00:56:18 +03:00
jussisaurio
1ae8d28669 Use same move_to() function for tables and indexes 2024-10-06 00:51:14 +03:00
jussisaurio
37f877109e Reduce duplication in btree.rs 2024-10-06 00:39:50 +03:00
jussisaurio
bb1c8b65e8 fmt 2024-10-06 00:22:12 +03:00
jussisaurio
dde10d2dd7 Better EXPLAIN QUERY PLAN for Operator::Search 2024-10-06 00:19:56 +03:00
jussisaurio
47534cb8df Get rid of Seekrowid operator in favor of a unified Search operator 2024-10-06 00:11:38 +03:00
jussisaurio
d3e797f59e rewind_labels was renamed to scan_loop_body_labels 2024-10-05 18:27:18 +03:00
jussisaurio
d22dbe9840 remove garbage comment 2024-10-05 18:25:04 +03:00
jussisaurio
3826d4e1ff Add comment about code duplication 2024-10-05 18:25:04 +03:00
jussisaurio
d2233d69d3 Dont assume the rowid is the second column - it's the last 2024-10-05 18:25:04 +03:00
jussisaurio
9169f6e39b same thing 2024-10-05 18:25:04 +03:00
jussisaurio
fe90aacd35 Handle CursorResult in deferred seek 2024-10-05 18:25:04 +03:00
jussisaurio
3a11887122 fixerinos 2024-10-05 18:25:04 +03:00
jussisaurio
ed19f47762 fix 2024-10-05 18:25:04 +03:00
jussisaurio
ff236c7781 Fix not advancing the cell index of pages 2024-10-05 18:25:04 +03:00
jussisaurio
99871bbeea yield on io 2024-10-05 18:25:04 +03:00
jussisaurio
e118b70127 fmt 2024-10-05 18:25:04 +03:00
jussisaurio
3d56fbd91c stuff 2024-10-05 18:25:04 +03:00
jussisaurio
f02da18acd index scan wip foo doesnt work yet 2024-10-05 18:25:04 +03:00
jussisaurio
d3015ad854 Merge 'Fix NextAwait's next instruction; rename rewind_labels' from Arpit Saxena 
Closes #351
NextAwait's next instruction was pointing to RewindAwait earlier which
is not current. The instruction now points to the instruction after
RewindAwait, which will be the loop body. Hence, rename rewind_labels to
scan_loop_body_labels to make it clearer

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #354
 2024-10-05 18:23:37 +03:00
Lauri Virtanen
9e80a0c4a8 Add randomblob(N) scalar function 
Relates to issue #144
 2024-10-03 00:05:23 +03:00
Pekka Enberg
b7656c265c Fix source formatting with cargo fmt... 2024-10-02 11:06:38 +03:00
Pekka Enberg
6fcd818160 Merge 'Add unhex(X) scalar function' from Lauri Virtanen 
This commit adds `unhex(X)` scalar function. Function with `unhex(X,Y)`
two arguments is not implemented yet.
Relates to issue #144

Closes #353
 2024-10-02 11:01:15 +03:00
Arpit Saxena
47c8fd1964 Fix NextAwait's next instruction; rename rewind_labels 
Closes #351

NextAwait's next instruction was pointing to RewindAwait earlier which
is not current. The instruction now points to the instruction after
RewindAwait, which will be the loop body. Hence, rename rewind_labels to
scan_loop_body_labels to make it clearer
 2024-10-02 11:58:42 +05:30
Lauri Virtanen
adc6f9b6cd Add unhex(X) scalar function 
This commit adds `unhex(X)` scalar function. Function with `unhex(X,Y)`
two arguments is not implemented yet.

Relates to issue #144
 2024-09-30 00:13:43 +03:00
Lauri Virtanen
f612ead8a3 Add zeroblob(N) scalar function 
Relates to issue #144
 2024-09-29 23:39:53 +03:00
Arpit Saxena
b7debabd81 Wrap IoUring to ensure pending_ops is always correctly updated 
Adds a struct WrappedIOUring which contains a IoUring and a pending_ops
field. Entry submission and popping from the queue is done through
functions operating on it, which also maintains pending_ops count

NOTE: This is a bit weird/hacky since in get_completion we create a
CompletionQueue and just call its next(). If it were a normal iterator
it would always return the same first item. However it is a queue posing
as an iterator which makes this work.
 2024-09-28 00:10:05 +05:30
Arpit Saxena
a7d735d368 Maintain pending io_uring ops count for early return on run_once 
Add a `pending_ops` field to `InnerLinuxIO` struct which is incremented
for each operation submitted to the ring and decremented when they are
taken off the completion queue. With this, we can exit from run_once if
there are no pending operations. Otherwise, in that case, it would hang
indefinitely due to call of `ring.submit_and_wait(1)`
 2024-09-27 23:32:19 +05:30
JeanArhancet
7b5d0fcabd refactor: split min/max func 2024-09-25 17:51:49 +02:00
JeanArhancet
222fea9cc8 fix: cmp number/text 2024-09-25 17:39:37 +02:00
Lauri Virtanen
0597c048fc Better support for BLOBs 
- Limbo command line shell supports e.g. `SELECT x'616263';`
- `EXPLAIN SELECT x'616263';` lists the opcode

Missing:

- Command line shell not entirely compatible with SQLite when blobs have
  non-printable characters in the middle (e.g. `SELECT x'610062';`)
- Python bindings not supported (incoming soon)
 2024-09-22 17:55:00 +03:00
Pekka Enberg
5aeb18f474 Fix source formatting with cargo fmt 2024-09-22 08:52:32 -04:00
baishen
5c00c576a5 Add support for hex scalar function 2024-09-22 08:50:08 -04:00
jussisaurio
c738674c5d Merge 'fix: cmp integer/float' from Jean Arhancet 
Closes https://github.com/penberg/limbo/issues/339

Reviewed-by: Lauri Virtanen <pere-altea@hotmail.com>

Closes #340
 2024-09-21 18:06:04 +03:00
JeanArhancet
41c1708ee7 fix: cmp integer/float 2024-09-21 16:10:42 +02:00
Lauri Virtanen
67573e12e5 Add typeof(X) scalar function 2024-09-21 15:56:29 +03:00
김선우
bbd1eea9ae Add support for parenthesized conditions 2024-09-21 16:03:49 +09:00
Pekka Enberg
b67640c437 Merge 'core: constraint check uniqueness' from Pere Diaz Bou 
```bash
limbo> insert into products values (1, 'asdf', 432);
Runtime error: UNIQUE constraint failed: products.id (19)
```

Closes #336
 2024-09-20 13:27:10 +03:00