aljaz/turso
1
0
Fork 0
mirror of https://github.com/aljazceru/turso.git synced 2025-12-27 13:04:20 +01:00
Code Issues Packages Projects Releases Wiki Activity
3,962 Commits 1 Branch 0 Tags
aece4e5442fe218f400a5952e64f933954f044e0
Commit Graph

2192 Commits

Author SHA1 Message Date
Pere Diaz Bou
aece4e5442 use seek instead of move_to in post balance delete 2025-04-15 10:59:49 +02:00
Pere Diaz Bou
651c00b4a8 allow index entry delete 2025-04-14 11:02:17 +02:00
Jussi Saurio
d286a56e15 refactor: fold Async/Await insns into a single instruction 2025-04-14 09:40:20 +03:00
Diego Reis
fd79ad2644 core/vdbe: Change is_btree to is_table in OpenEphemeral 2025-04-13 11:15:01 -03:00
Diego Reis
4c315e1bb6 core/vdbe: Update OpenEphemeral to use CreateBtreeFlags 2025-04-13 11:13:25 -03:00
Diego Reis
135330b736 core/pager: Fix page handling issue due change in wal type 2025-04-13 11:10:06 -03:00
Diego Reis
035e6dcef4 core/vdbe: Fix logic error during btree creation 
I do thing we should change this 1,2 flag to 0,1 or just an enum, to be more rustacean. The current state can be very misleading
 2025-04-13 11:10:06 -03:00
Diego Reis
61c324cca5 core/vdbe: Add missing work to get cursor and transient table usable 2025-04-13 11:10:06 -03:00
Diego Reis
bcac1fe778 core/vdbe: Rename page_io to db_file in OpenEphemeral 2025-04-13 11:10:06 -03:00
Diego Reis
09d83aadf3 Fix dumb conflict errors 2025-04-13 11:10:06 -03:00
Diego Reis
d9bf383507 core/io: Untie MemoryIO's lifetime of the IO layer 2025-04-13 11:10:06 -03:00
Diego Reis
79f8b83cbe Fix dumb clippy errors 2025-04-13 11:10:06 -03:00
Diego Reis
66e12e1c2d core/vdbe: Create OpenEphemeral bytecode 
"Open a new cursor P1 to a transient table. The cursor is always opened read/write even if the main database is read-only. The ephemeral table is deleted automatically when the cursor is closed.

If the cursor P1 is already opened on an ephemeral table, the table is cleared (all content is erased)."

There is still some work to do, but this is a basic setup
 2025-04-13 11:10:06 -03:00
Diego Reis
e5144bb6a9 core/storage: Create FileMemoryStorage 
This is basically a copy of DatabaseStorage but outside the fs compilation flag, this way, we can access MemoryIO regardless the storage medium.
 2025-04-13 11:10:06 -03:00
Diego Reis
b519509349 core/io: Add internal in-memory MemoryIO to all IO layers 
Honestly I don't have 100% sure if this is a good idea, the reasoning is that in any IO we'll want to do memory only operations like creating tables etc, so may want a common way to access it
 2025-04-13 11:10:05 -03:00
Diego Reis
65d4c68cf2 core/pager: Wrap wal with Option 2025-04-13 11:10:05 -03:00
Jussi Saurio
8e601959ad Merge 'Fuzz fix some operations' from Pedro Muniz 
Me and @diegoreis42 were late at night, running the new fuzzer and saw
some errors with `Bit-Not` and `Not` operations. Added some tests as
well for those fixes.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1328
 2025-04-13 17:02:19 +03:00
Jussi Saurio
cbe3500b7e Merge 'Code clean-ups' from Diego Reis 
While developing I found that some things could be improved :)

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1325
 2025-04-13 17:02:12 +03:00
Jussi Saurio
b51f9bba23 Merge 'Return null when parameter is unbound' from Levy A. 
Fix #1323.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1324
 2025-04-13 17:02:03 +03:00
pedrocarlo
1297cb107c bit-not and boolean-not 
Co-authored-by: Diego Reis <79876389+diegoreis42@users.noreply.github.com>
 2025-04-13 02:45:12 -03:00
Diego Reis
51eb2af06a core(refactor): Add CreateBTreeFlags 
Passing 1s and 0s with comments is not rustacean, and since we already follow the pattern of struct flags in other sections of the codebase it's better use it here too.
 2025-04-13 01:46:30 -03:00
Diego Reis
da484b33aa core/storage: Add asserts to check invariants for current_page 
While developing, I had an underflow issue during the casting: `current as usize`. Which could be found way faster with proper asserts.
 2025-04-12 22:29:29 -03:00
Levy A.
d210ee1497 cargo fmt 2025-04-12 18:55:48 -03:00
Diego Reis
ff7a4e8297 core: Change always falseness of equivalence between variables expressions to be only on anonymous variables 
Named variables are compared by name
 2025-04-12 18:04:15 -03:00
Diego Reis
64f8aca823 core: Improve readability of Affinity's doc comment 2025-04-12 17:55:57 -03:00
Levy A.
5c0b112125 fix: return null when parameter is unbound 2025-04-12 17:43:04 -03:00
Diego Reis
db0f07499d core/translate: Fix naive comparison between Binary expressions during register optimization 2025-04-12 16:39:30 -03:00
Diego Reis
73764e198e core: Fix equivalence between variable expressions to be always false 
Since until the bind to a value they are treated as NULL. https://sqlite.org/lang_expr.html#varparam
 2025-04-12 16:34:39 -03:00
Jussi Saurio
6bea4de30f Check that index seek key members are not null 2025-04-11 17:22:46 +03:00
Jussi Saurio
2cbb903b06 Add doc comments to SeekOp 2025-04-11 13:45:10 +03:00
Jussi Saurio
029a0c86b2 btree: remove IterationState 
iteration direction must be known when seeking, and transitively
when using move_to() since seek() uses it, but otherwise IterationState
just brings way too much noise to the code -- it was meant to encode
invariants about how a cursor can be iterated, but it's not worth it.

iteration direction for seek()/move_to() can be inferred from the
SeekOp:

GE/GT/EQ: forward
LT/LE: backward

and get_next_record()/get_prev_record() already have different
logic for their respective iteration directions.
 2025-04-11 13:40:51 +03:00
Pekka Enberg
e3a4400329 Merge 'Multi column indexes + index seek refactor' from Jussi Saurio 
# Multi column indexes + index seek refactor
## PR reader guide
I would say mostly you should just focus on the content of
`optimizer.rs` and `plan.rs` because the rest is just small type
changes, or in the case of `main_loop.rs`, a bunch of logic was just
moved out of there and rewritten.
## New feature - multi column index seeks
This PR adds support for utilizing multi-column indexes properly, i.e.
using as many columns in the seek key as possible. Previously, we only
used max one column per index. I've modified the existing compound index
seek fuzz test to use this functionality.
## Refactoring of index seek related logic
This PR moves a lot of index seek related logic out of `main_loop.rs`
into `optimizer.rs` and `plan.rs` and introduces a bunch of helper
structures to model finding and using an index to perform a seek + scan.
## Examples
Here are some examples of multi-column seeks:
### Example table setup:
```sql
sqlite> CREATE TABLE t(a,b,c,d,e);
sqlite> CREATE INDEX abc ON t (a,b,c);
-- create 10000 rows with random values between 0-9 for all columns
sqlite >INSERT INTO t SELECT ABS(RANDOM() % 10),ABS(RANDOM() % 10),ABS(RANDOM() % 10),ABS(RANDOM() % 10),ABS(RANDOM() % 10) FROM generate_series(1,10000,1);
```
### Example bytecode plans, results and timings vs main branch:
```sql
limbo> EXPLAIN SELECT * FROM t WHERE a = 5 and b = 6 and c = 7;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     20    0                    0   Start at 20
1     OpenReadAsync      0     2     0                    0   table=t, root=2
2     OpenReadAwait      0     0     0                    0
3     OpenReadAsync      1     3     0                    0   table=abc, root=3
4     OpenReadAwait      0     0     0                    0
5     Integer            5     6     0                    0   r[6]=5
6     Integer            6     7     0                    0   r[7]=6
7     Integer            7     8     0                    0   r[8]=7
8     SeekGE             1     19    6                    0   key=[6..8]
9       IdxGT            1     19    6                    0   key=[6..8]
10      DeferredSeek     1     0     0                    0
11      Column           0     0     1                    0   r[1]=t.a
12      Column           0     1     2                    0   r[2]=t.b
13      Column           0     2     3                    0   r[3]=t.c
14      Column           0     3     4                    0   r[4]=t.d
15      Column           0     4     5                    0   r[5]=t.e
16      ResultRow        1     5     0                    0   output=r[1..5]
17    NextAsync          1     0     0                    0
18    NextAwait          1     9     0                    0
19    Halt               0     0     0                    0
20    Transaction        0     0     0                    0   write=false
21    Goto               0     1     0                    0

limbo> SELECT * FROM t WHERE a = 5 and b = 6 and c = 7;
5|6|7|9|9
5|6|7|4|7
5|6|7|3|2
5|6|7|3|7
5|6|7|5|2
5|6|7|5|3
5|6|7|9|7

runtime (debug build, this branch): total: 2 ms (this includes parsing/coloring of cli app)
runtime (debud build, main branch): total: 67 ms (this includes parsing/coloring of cli app)

```
```sql
limbo> EXPLAIN SELECT * FROM t WHERE a = 5 and b = 6 and c < 7;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     21    0                    0   Start at 21
1     OpenReadAsync      0     2     0                    0   table=t, root=2
2     OpenReadAwait      0     0     0                    0
3     OpenReadAsync      1     3     0                    0   table=abc, root=3
4     OpenReadAwait      0     0     0                    0
5     Integer            5     6     0                    0   r[6]=5
6     Integer            6     7     0                    0   r[7]=6
7     Null               0     8     0                    0   r[8]=NULL
8     SeekGT             1     20    6                    0   key=[6..8]
9       Integer          7     8     0                    0   r[8]=7
10      IdxGE            1     20    6                    0   key=[6..8]
11      DeferredSeek     1     0     0                    0
12      Column           0     0     1                    0   r[1]=t.a
13      Column           0     1     2                    0   r[2]=t.b
14      Column           0     2     3                    0   r[3]=t.c
15      Column           0     3     4                    0   r[4]=t.d
16      Column           0     4     5                    0   r[5]=t.e
17      ResultRow        1     5     0                    0   output=r[1..5]
18    NextAsync          1     0     0                    0
19    NextAwait          1     10    0                    0
20    Halt               0     0     0                    0
21    Transaction        0     0     0                    0   write=false
22    Goto               0     1     0                    0

limbo> SELECT * FROM t WHERE a = 5 and b = 6 and c < 7;
5|6|0|0|3
5|6|0|5|1
5|6|0|3|1
5|6|0|6|3
5|6|0|8|1
5|6|0|2|7
5|6|0|9|9
5|6|0|5|3
5|6|0|4|2
5|6|0|4|2
5|6|0|0|2
5|6|0|7|2
5|6|1|8|5
5|6|1|7|5
5|6|1|7|2
5|6|1|1|2
5|6|1|6|5
5|6|1|1|5
5|6|1|5|7
5|6|1|1|9
5|6|1|4|3
5|6|1|1|2
5|6|1|2|2
5|6|1|4|4
5|6|1|9|6
5|6|1|2|5
5|6|1|2|4
5|6|1|7|1
5|6|2|0|9
5|6|2|6|9
5|6|2|4|5
5|6|2|9|3
5|6|2|5|2
5|6|2|9|0
5|6|2|7|1
5|6|3|6|5
5|6|3|8|5
5|6|3|5|4
5|6|3|5|2
5|6|3|1|1
5|6|3|2|0
5|6|3|9|3
5|6|3|6|9
5|6|3|7|6
5|6|3|3|5
5|6|3|0|8
5|6|3|6|4
5|6|4|1|1
5|6|4|9|8
5|6|4|3|7
5|6|4|1|3
5|6|4|8|9
5|6|4|9|7
5|6|4|7|9
5|6|4|8|8
5|6|4|3|1
5|6|4|2|6
5|6|4|5|7
5|6|4|2|6
5|6|4|4|3
5|6|5|2|4
5|6|5|6|7
5|6|5|3|8
5|6|5|7|8
5|6|5|9|6
5|6|5|2|7
5|6|5|1|7
5|6|5|0|6
5|6|6|2|4
5|6|6|9|4
5|6|6|4|9
5|6|6|5|6
5|6|6|2|2
5|6|6|0|6

runtime (debug build, this branch): total: 9 ms (this includes parsing/coloring of cli app)
runtime (debug build, main branch): total: 71 ms (this includes parsing/coloring of cli app)

```
```sql
limbo> EXPLAIN SELECT * FROM t WHERE a = 5 and b = 6 and c < 7 ORDER BY a desc, b desc, c desc;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     20    0                    0   Start at 20
1     OpenReadAsync      0     2     0                    0   table=t, root=2
2     OpenReadAwait      0     0     0                    0
3     OpenReadAsync      1     3     0                    0   table=abc, root=3
4     OpenReadAwait      0     0     0                    0
5     Integer            5     6     0                    0   r[6]=5
6     Integer            6     7     0                    0   r[7]=6
7     Integer            7     8     0                    0   r[8]=7
8     SeekLT             1     19    6                    0   key=[6..8]
9       IdxLT            1     19    6                    0   key=[6..7]
10      DeferredSeek     1     0     0                    0
11      Column           0     0     1                    0   r[1]=t.a
12      Column           0     1     2                    0   r[2]=t.b
13      Column           0     2     3                    0   r[3]=t.c
14      Column           0     3     4                    0   r[4]=t.d
15      Column           0     4     5                    0   r[5]=t.e
16      ResultRow        1     5     0                    0   output=r[1..5]
17    PrevAsync          1     0     0                    0
18    PrevAwait          1     0     0                    0
19    Halt               0     0     0                    0
20    Transaction        0     0     0                    0   write=false
21    Goto               0     1     0                    0

limbo> SELECT * FROM t WHERE a = 5 and b = 6 and c < 7 ORDER BY a desc, b desc, c desc;
5|6|6|0|6
5|6|6|2|2
5|6|6|5|6
5|6|6|4|9
5|6|6|9|4
5|6|6|2|4
5|6|5|0|6
5|6|5|1|7
5|6|5|2|7
5|6|5|9|6
5|6|5|7|8
5|6|5|3|8
5|6|5|6|7
5|6|5|2|4
5|6|4|4|3
5|6|4|2|6
5|6|4|5|7
5|6|4|2|6
5|6|4|3|1
5|6|4|8|8
5|6|4|7|9
5|6|4|9|7
5|6|4|8|9
5|6|4|1|3
5|6|4|3|7
5|6|4|9|8
5|6|4|1|1
5|6|3|6|4
5|6|3|0|8
5|6|3|3|5
5|6|3|7|6
5|6|3|6|9
5|6|3|9|3
5|6|3|2|0
5|6|3|1|1
5|6|3|5|2
5|6|3|5|4
5|6|3|8|5
5|6|3|6|5
5|6|2|7|1
5|6|2|9|0
5|6|2|5|2
5|6|2|9|3
5|6|2|4|5
5|6|2|6|9
5|6|2|0|9
5|6|1|7|1
5|6|1|2|4
5|6|1|2|5
5|6|1|9|6
5|6|1|4|4
5|6|1|2|2
5|6|1|1|2
5|6|1|4|3
5|6|1|1|9
5|6|1|5|7
5|6|1|1|5
5|6|1|6|5
5|6|1|1|2
5|6|1|7|2
5|6|1|7|5
5|6|1|8|5
5|6|0|7|2
5|6|0|0|2
5|6|0|4|2
5|6|0|4|2
5|6|0|5|3
5|6|0|9|9
5|6|0|2|7
5|6|0|8|1
5|6|0|6|3
5|6|0|3|1
5|6|0|5|1
5|6|0|0|3

runtime (debug build, this branch): total: 9 ms (this includes parsing/coloring of cli app)
runtime (debug build, main branch): total: 71 ms (this includes parsing/coloring of cli app)
```

Closes #1288
 2025-04-11 09:36:25 +03:00
Pekka Enberg
d67e1b604b Merge 'Added 'likelihood' scalar function' from Sachin Kumar Singh 
The `likelihood(X,Y)` function returns argument X unchanged. The value Y
in likelihood(X,Y) must be a floating point constant between 0.0 and
1.0, inclusive.
```
sqlite> explain SELECT likelihood(42, 0.0);
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     6     0                    0   Start at 6
1     Once           0     3     0                    0
2     Integer        42    2     0                    0   r[2]=42
3     Copy           2     1     0                    0   r[1]=r[2]
4     ResultRow      1     1     0                    0   output=r[1]
5     Halt           0     0     0                    0
6     Goto           0     1     0                    0
```
```
limbo> explain SELECT likelihood(42, 0.0);
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     4     0                    0   Start at 4
1     Copy               2     1     0                    0   r[1]=r[2]
2     ResultRow          1     1     0                    0   output=r[1]
3     Halt               0     0     0                    0
4     Integer            42    2     0                    0   r[2]=42
5     Goto               0     1     0                    0
```

Closes #1303
 2025-04-11 09:34:36 +03:00
Pekka Enberg
13516fd53d Merge 'feat: Add timediff data and time function' from Sachin Kumar Singh 
This PR implemets the `timediff(A,B)` function, which returns a string
that describes the amount of time that must be added to B in order to
reach time A. I used sqlite's timediff function for format reference:
https://github.com/sqlite/sqlite/blob/master/src/date.c#L1694
Op-codes seems to be in order:
```
limbo> explain SELECT timediff('12:30:45.123', '12:30:44.987');
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     6     0                    0   Start at 6
1     String8            0     2     0     12:30:45.123   0   r[2]='12:30:45.123'
2     String8            0     3     0     12:30:44.987   0   r[3]='12:30:44.987'
3     Function           0     2     1     timediff       0   r[1]=func(r[2..3])
4     ResultRow          1     1     0                    0   output=r[1]
5     Halt               0     0     0                    0
6     Goto               0     1     0                    0
```
```
sqlite> explain SELECT timediff('12:30:45.123', '12:30:44.987');
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     8     0                    0   Start at 8
1     Once           0     5     0                    0
2     String8        0     3     0     12:30:45.123   0   r[3]='12:30:45.123'
3     String8        0     4     0     12:30:44.987   0   r[4]='12:30:44.987'
4     Function       3     3     2     timediff(2)    0   r[2]=func(r[3..4])
5     Copy           2     1     0                    0   r[1]=r[2]
6     ResultRow      1     1     0                    0   output=r[1]
7     Halt           0     0     0                    0
8     Goto           0     1     0                    0
```
My first PR, I just followed the [contributing guides](https://github.co
m/tursodatabase/limbo/blob/main/CONTRIBUTING.md) and started.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1302
 2025-04-11 09:34:04 +03:00
Sachin Singh
23ab387143 handle formatting issues 2025-04-11 09:59:27 +05:30
Sachin Singh
01fa02364d correctly handle edge cases 2025-04-11 08:34:29 +05:30
Sachin Singh
482e93bfd0 feat: add likelihood scalar function 2025-04-11 05:54:23 +05:30
Sachin Singh
b7acfa490c feat: add timediff data and time function 2025-04-11 04:30:57 +05:30
Pere Diaz Bou
745c2b92d0 unnecessary dirty set on overwrite 2025-04-10 22:24:15 +02:00
Pere Diaz Bou
038d78f096 overwrite when payload is equal size as current cell only 
Prevoiusly we would overwrite even though size less than cell size. This
was wrong because it didn't update any fragment size or free blocks it
could. To be safe let's just overwrite only if local size is the same
amount.
 2025-04-10 22:24:15 +02:00
Pere Diaz Bou
506c1a236c find_free_cell fix use of no_offset writes 2025-04-10 22:24:15 +02:00
Pekka Enberg
ef893da6c7 Merge 'core/btree: Add PageContent::new() helper' from Pekka Enberg 
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1294
 2025-04-10 20:53:41 +03:00
Pekka Enberg
a27126cd05 Merge 'B-Tree code cleanups' from Pekka Enberg 
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1290
 2025-04-10 20:53:33 +03:00
Jussi Saurio
4daad0a858 Fix bug: accidentally skipped index selection for other tables except first found 2025-04-10 18:57:14 +03:00
Pere Diaz Bou
62d0febdb6 panic on corruption 2025-04-10 16:01:24 +02:00
Pere Diaz Bou
8e93471d00 fix cell index selection while balancing 
Cell index doesn't move in `move_to` unless we don't need to check next
cell. On the other hand, with rightmost pointer, we advance cell index
by 1 even though where we are moving to was to that page
 2025-04-10 16:01:24 +02:00
Jussi Saurio
457bded14d optimizer: refactor optimizer to support multicolumn index scans 2025-04-10 15:53:02 +03:00
Jussi Saurio
afad06fb23 vdbe/explain: add key info to Seek/Idx insns 2025-04-10 15:06:45 +03:00
Jussi Saurio
60a13c129f io/linux: make syscallio the default (io_uring is really slow) 2025-04-10 13:32:26 +03:00
Pekka Enberg
53633e8b6f core/btree: Add PageContent::new() helper 2025-04-10 13:14:38 +03:00