Pere Diaz Bou
8d1d2d36cc
core/linux: aligned wal header read
2024-11-12 17:53:59 +00:00
Pere Diaz Bou
48f0e72e14
checkpoint on drop connection
2024-11-12 17:03:30 +01:00
Pere Diaz Bou
807496a146
core: load pages if not loaded
...
Since pages are now tracked with a single centralized structure, it is
possible for a page to have been unloaded a be kept in memory for
metadata purposes like going back in the stack.
Using the example of going to a parent page which was unloaded for
whatever reason: in this case we need to check if it's loaded, if not,
we load it. Locked still means the same thing, loaded just means the
contents of a page are present in memory and if they are present, they
must be in cache.
2024-11-12 14:44:25 +01:00
Pere Diaz Bou
cd2b61d838
btree: cursor with lineal stack structure
...
Removed MemPage from the code in favor of an array to encode the stack
of the cursor. This is both simpler and better in terms of memory
access.
2024-11-12 10:21:22 +01:00
Pere Diaz Bou
ce8378793e
btree: run until finished or IO + bug fixes
2024-11-11 17:03:15 +01:00
Pere Diaz Bou
74974d07e7
clippy fix
2024-11-08 11:14:29 +01:00
Pere Diaz Bou
e5836df160
write state machine
...
Returning on ongoing IO requires to save the current state, therefore a
state machine makes sense for write operations. This information is
encoded in WriteInfo/WriteState.
2024-11-08 11:10:49 +01:00
Pere Diaz Bou
ecfcd8835f
rename get_page -> get_current_page
2024-11-07 15:37:21 +01:00
Pere Diaz Bou
7d8fc80f4b
use len instead of size
2024-11-07 14:01:35 +01:00
Pere Diaz Bou
ce1367b76a
move wal creation
2024-11-07 13:51:02 +01:00
Pere Diaz Bou
218c32e0e6
remove unix import in windoews
2024-11-07 13:34:48 +01:00
Pere Diaz Bou
0b46648c0e
write wal header on init
2024-11-07 13:29:57 +01:00
Pere Diaz Bou
0d857661f2
windows complete completion
2024-11-07 09:57:24 +01:00
Pere Diaz Bou
2514287263
windows fix
2024-11-06 18:11:15 +01:00
Pere Diaz Bou
cef78d54de
fix generic
2024-11-06 17:53:14 +01:00
Pere Diaz Bou
eb8c462c5f
fix io submission on cacheflush
2024-11-06 16:25:42 +00:00
Pere Diaz Bou
8eb3c89227
wasm,sim fixes
2024-11-05 15:41:30 +01:00
Pere Diaz Bou
a85d599c65
state machine cacheflush
2024-11-05 15:29:54 +01:00
Pere Diaz Bou
da7717edfb
fix generic io match
2024-11-05 15:29:54 +01:00
Pere Diaz Bou
5207e49399
remove extra borrow mut
2024-11-05 15:29:54 +01:00
Pere Diaz Bou
0f4270b48f
rebase submit entry
2024-11-05 15:29:54 +01:00
Pere Diaz Bou
70a4ccd8bb
fix linux completion match
2024-11-05 15:29:54 +01:00
Pere Diaz Bou
fc65c5096d
cacheflush state machine
2024-11-05 15:29:54 +01:00
Pere Diaz Bou
129cc1cd6d
fix open_file generic
2024-11-05 15:29:54 +01:00
Pere Diaz Bou
f66e3925f3
fix imports
2024-11-05 15:29:54 +01:00
Pere Diaz Bou
f009eb35c6
suspendable checkpoint
2024-11-05 15:29:54 +01:00
Pere Diaz Bou
c0e51c4ca6
wip wal
2024-11-05 15:29:53 +01:00
jussisaurio
daf5863932
manual fixes based on clippy suggestions
2024-10-13 12:19:04 +03:00
jussisaurio
f634e7f7a3
clippy fix
2024-10-13 12:08:54 +03:00
jussisaurio
b3c57d5691
core/translate: (refactor) use btreetablereference struct instead of tuple
2024-10-13 11:25:33 +03:00
Pekka Enberg
bf2e254a21
Merge 'core/translate/optimizer: eliminate unnecessary ORDER BY if result set is already ordered' from Jussi Saurio
...
Built on top of, and currently targets, #350 , _not_ main
Closes #365
Examples:
```
limbo> explain select u.first_name, p.name from users u join products p on u.id = p.id order by u.id;
addr opcode p1 p2 p3 p4 p5 comment
---- ----------------- ---- ---- ---- ------------- -- -------
0 Init 0 15 0 0 Start at 15
1 OpenReadAsync 0 2 0 0 table=u, root=2
2 OpenReadAwait 0 0 0 0
3 OpenReadAsync 1 3 0 0 table=p, root=3
4 OpenReadAwait 0 0 0 0
5 RewindAsync 0 0 0 0
6 RewindAwait 0 14 0 0 Rewind table u
7 RowId 0 1 0 0 r[1]=u.rowid
8 SeekRowid 1 1 12 0 if (r[1]!=p.rowid) goto 12
9 Column 0 1 2 0 r[2]=u.first_name
10 Column 1 1 3 0 r[3]=p.name
11 ResultRow 2 2 0 0 output=r[2..3]
12 NextAsync 0 0 0 0
13 NextAwait 0 7 0 0
14 Halt 0 0 0 0
15 Transaction 0 0 0 0
16 Goto 0 1 0 0
```
```
limbo> explain select * from users where age > 80 order by age limit 5;
addr opcode p1 p2 p3 p4 p5 comment
---- ----------------- ---- ---- ---- ------------- -- -------
0 Init 0 23 0 0 Start at 23
1 OpenReadAsync 0 2 0 0 table=users, root=2
2 OpenReadAwait 0 0 0 0
3 OpenReadAsync 1 274 0 0 table=age_idx, root=274
4 OpenReadAwait 0 0 0 0
5 Integer 80 1 0 0 r[1]=80
6 SeekGT 1 22 1 0
7 DeferredSeek 1 0 0 0
8 RowId 0 2 0 0 r[2]=users.rowid
9 Column 0 1 3 0 r[3]=users.first_name
10 Column 0 2 4 0 r[4]=users.last_name
11 Column 0 3 5 0 r[5]=users.email
12 Column 0 4 6 0 r[6]=users.phone_number
13 Column 0 5 7 0 r[7]=users.address
14 Column 0 6 8 0 r[8]=users.city
15 Column 0 7 9 0 r[9]=users.state
16 Column 0 8 10 0 r[10]=users.zipcode
17 Column 0 9 11 0 r[11]=users.age
18 ResultRow 2 10 0 0 output=r[2..11]
19 DecrJumpZero 12 22 0 0 if (--r[12]==0) goto 22
20 NextAsync 1 0 0 0
21 NextAwait 1 7 0 0
22 Halt 0 0 0 0
23 Transaction 0 0 0 0
24 Integer 5 12 0 0 r[12]=5
25 Goto 0 1 0 0
```
Closes #366
2024-10-13 10:12:28 +03:00
Pekka Enberg
fdf3b0d16b
Merge '(core): Primary key index scans and single-column secondary index scans' from Jussi Saurio
...
This PR adds an index on `users.age` to `testing.db`, and support for
indexed lookups. Only single-column ascending indexes are currently
supported.
This PR also gets rid of `Operator::Seekrowid` in favor of
`Operator::Search` which handles all non-full-table-scan searches: 1.
integer primary key (rowid) point queries 2. integer primary key index
scans, and 3. secondary index scans.
examples:
```
limbo> select first_name, age from users where age > 90 limit 10;
Miranda|90
Sarah|90
Justin|90
Justin|90
John|90
Jeremy|90
Stephanie|90
Joshua|90
Jenny|90
Jennifer|90
limbo> explain query plan select first_name, age from users where age > 90 limit 10;
QUERY PLAN
`--TAKE 10
`--PROJECT first_name, age
| `--SEARCH users USING INDEX age_idx
limbo> explain select first_name, age from users where age > 90 limit 10;
addr opcode p1 p2 p3 p4 p5 comment
---- ----------------- ---- ---- ---- ------------- -- -------
0 Init 0 15 0 0 Start at 15
1 OpenReadAsync 0 2 0 0 table=users, root=2
2 OpenReadAwait 0 0 0 0
3 OpenReadAsync 1 274 0 0 table=age_idx, root=274
4 OpenReadAwait 0 0 0 0
5 Integer 90 1 0 0 r[1]=90
6 SeekGT 1 14 1 0
7 DeferredSeek 1 0 0 0
8 Column 0 1 2 0 r[2]=users.first_name
9 Column 0 9 3 0 r[3]=users.age
10 ResultRow 2 2 0 0 output=r[2..3]
11 DecrJumpZero 4 14 0 0 if (--r[4]==0) goto 14
12 NextAsync 1 0 0 0
13 NextAwait 1 7 0 0
14 Halt 0 0 0 0
15 Transaction 0 0 0 0
16 Integer 10 4 0 0 r[4]=10
17 Goto 0 1 0 0
```
Sqlite version:
```
sqlite> explain select first_name, age from users where age > 90 limit 10;
addr opcode p1 p2 p3 p4 p5 comment
---- ------------- ---- ---- ---- ------------- -- -------------
0 Init 0 13 0 0 Start at 13
1 Integer 10 1 0 0 r[1]=10; LIMIT counter
2 OpenRead 0 2 0 10 0 root=2 iDb=0; users
3 OpenRead 1 274 0 k(2,,) 0 root=274 iDb=0; age_idx
4 Integer 90 2 0 0 r[2]=90
5 SeekGT 1 12 2 1 0 key=r[2]
6 DeferredSeek 1 0 0 0 Move 0 to 1.rowid if needed
7 Column 0 1 3 0 r[3]= cursor 0 column 1
8 Column 1 0 4 0 r[4]= cursor 1 column 0
9 ResultRow 3 2 0 0 output=r[3..4]
10 DecrJumpZero 1 12 0 0 if (--r[1])==0 goto 12
11 Next 1 6 0 0
12 Halt 0 0 0 0
13 Transaction 0 0 3 0 1 usesStmtJournal=0
14 Goto 0 1 0 0
```
---
´Seek` instructions are also now supported for primary key rowid
searches:
```
limbo> select id, first_name from users where id > 9995;
9996|Donald
9997|Ruth
9998|Dorothy
9999|Gina
10000|Nicole
limbo> explain query plan select id, first_name from users where id > 9995;
QUERY PLAN
`--PROJECT id, first_name
`--SEARCH users USING INTEGER PRIMARY KEY (rowid=?)
limbo> explain select id, first_name from users where id > 9995;
addr opcode p1 p2 p3 p4 p5 comment
---- ----------------- ---- ---- ---- ------------- -- -------
0 Init 0 11 0 0 Start at 11
1 OpenReadAsync 0 2 0 0 table=users, root=2
2 OpenReadAwait 0 0 0 0
3 Integer 9995 1 0 0 r[1]=9995
4 SeekGT 0 10 1 0
5 RowId 0 2 0 0 r[2]=users.rowid
6 Column 0 1 3 0 r[3]=users.first_name
7 ResultRow 2 2 0 0 output=r[2..3]
8 NextAsync 0 0 0 0
9 NextAwait 0 5 0 0
10 Halt 0 0 0 0
11 Transaction 0 0 0 0
12 Goto 0 1 0 0
```
sqlite:
```
sqlite> explain select id, first_name from users where id > 9995;
addr opcode p1 p2 p3 p4 p5 comment
---- ------------- ---- ---- ---- ------------- -- -------------
0 Init 0 8 0 0 Start at 8
1 OpenRead 0 2 0 2 0 root=2 iDb=0; users
2 SeekGT 0 7 1 0 key=r[1]; pk
3 Rowid 0 2 0 0 r[2]=users.rowid
4 Column 0 1 3 0 r[3]= cursor 0 column 1
5 ResultRow 2 2 0 0 output=r[2..3]
6 Next 0 3 0 0
7 Halt 0 0 0 0
8 Transaction 0 0 3 0 1 usesStmtJournal=0
9 Integer 9995 1 0 0 r[1]=9995
10 Goto 0 1 0 0
```
---
More complex example with a join that uses both a rowid lookup and a
secondary index scan:
```
limbo> explain query plan select u.first_name, p.name from users u join products p on u.id = p.id and u.age > 70;
QUERY PLAN
`--PROJECT u.first_name, p.name
`--JOIN
| |--SEARCH u USING INDEX age_idx
| `--SEARCH p USING INTEGER PRIMARY KEY (rowid=?)
limbo> explain select u.first_name, p.name from users u join products p on u.id = p.id and u.age > 70;
addr opcode p1 p2 p3 p4 p5 comment
---- ----------------- ---- ---- ---- ------------- -- -------
0 Init 0 18 0 0 Start at 18
1 OpenReadAsync 0 2 0 0 table=u, root=2
2 OpenReadAwait 0 0 0 0
3 OpenReadAsync 1 274 0 0 table=age_idx, root=274
4 OpenReadAwait 0 0 0 0
5 OpenReadAsync 2 3 0 0 table=p, root=3
6 OpenReadAwait 0 0 0 0
7 Integer 70 1 0 0 r[1]=70
8 SeekGT 1 17 1 0
9 DeferredSeek 1 0 0 0
10 RowId 0 2 0 0 r[2]=u.rowid
11 SeekRowid 2 2 15 0 if (r[2]!=p.rowid) goto 15
12 Column 0 1 3 0 r[3]=u.first_name
13 Column 2 1 4 0 r[4]=p.name
14 ResultRow 3 2 0 0 output=r[3..4]
15 NextAsync 1 0 0 0
16 NextAwait 1 9 0 0
17 Halt 0 0 0 0
18 Transaction 0 0 0 0
19 Goto 0 1 0 0
```
sqlite:
```
sqlite> explain select u.first_name, p.name from users u join products p on u.id = p.id and u.age > 70;
addr opcode p1 p2 p3 p4 p5 comment
---- ------------- ---- ---- ---- ------------- -- -------------
0 Init 0 14 0 0 Start at 14
1 OpenRead 0 2 0 10 0 root=2 iDb=0; users
2 OpenRead 2 274 0 k(2,,) 0 root=274 iDb=0; age_idx
3 OpenRead 1 3 0 2 0 root=3 iDb=0; products
4 Integer 70 1 0 0 r[1]=70
5 SeekGT 2 13 1 1 0 key=r[1]
6 DeferredSeek 2 0 0 0 Move 0 to 2.rowid if needed
7 IdxRowid 2 2 0 0 r[2]=rowid; users.rowid
8 SeekRowid 1 12 2 0 intkey=r[2]
9 Column 0 1 3 0 r[3]= cursor 0 column 1
10 Column 1 1 4 0 r[4]= cursor 1 column 1
11 ResultRow 3 2 0 0 output=r[3..4]
12 Next 2 6 0 0
13 Halt 0 0 0 0
14 Transaction 0 0 3 0 1 usesStmtJournal=0
15 Goto 0 1 0 0
```
Closes #350
2024-10-13 10:11:23 +03:00
jussisaurio
04ecbff7fc
Eliminate unnecessary ORDER BY if result set is already ordered
2024-10-10 23:43:39 +03:00
Arpit Saxena
28a603f56a
[sorter] Hold records in Vec instead of a BTreeMap
...
We now just insert them one after the other in the vector. When rewind
is called, the vector is sorted. Iterating is just taking elements from
the vector
2024-10-09 17:58:28 +05:30
jussisaurio
556f4b73c9
Refine edge case handling: add optional predicate to get_next_record()
2024-10-08 08:23:30 +03:00
jussisaurio
43038cb6aa
Handle seek() edge case with index seek
2024-10-08 07:45:21 +03:00
jussisaurio
572db69b5e
Add TODO comment about index corner case
2024-10-07 17:12:19 +03:00
jussisaurio
93a8110773
dont assume index key has rowid in the second column: its the last
2024-10-07 17:05:38 +03:00
jussisaurio
8563d620af
renaming
2024-10-07 17:03:50 +03:00
jussisaurio
fc71f2b32f
traverse index properly
...
interior index cells have values that are not in the leaves, e.g.
(interior: 3)
/ \
(leaf: 2) (leaf: 4)
so their values need to be emitted after the left subtree is emitted.
2024-10-07 13:04:03 +03:00
jussisaurio
e5cf052f07
Why do sqlite btree child keys have <= keys and not < keys
2024-10-06 23:48:59 +03:00
Lauri Virtanen
0ae1412193
Add instr(X,Y) scalar function
...
Relates to issue #144
2024-10-06 20:19:37 +03:00
jussisaurio
15a66ea662
single seek function in cursor trait
2024-10-06 09:21:15 +03:00
jussisaurio
6e7db36121
reorder
2024-10-06 00:58:32 +03:00
jussisaurio
af9a751d36
Single seek function
2024-10-06 00:56:18 +03:00
jussisaurio
1ae8d28669
Use same move_to() function for tables and indexes
2024-10-06 00:51:14 +03:00
jussisaurio
37f877109e
Reduce duplication in btree.rs
2024-10-06 00:39:50 +03:00
jussisaurio
bb1c8b65e8
fmt
2024-10-06 00:22:12 +03:00
jussisaurio
dde10d2dd7
Better EXPLAIN QUERY PLAN for Operator::Search
2024-10-06 00:19:56 +03:00
jussisaurio
47534cb8df
Get rid of Seekrowid operator in favor of a unified Search operator
2024-10-06 00:11:38 +03:00
