Commit Graph

4705 Commits

Author SHA1 Message Date
Preston Thorpe
8d05336522 Merge 'Fix affinity handling in MakeRecord' from Pekka Enberg
Closes #2966
2025-09-08 12:14:33 -04:00
Pekka Enberg
71a812ce55 Merge 'Fix infinite loop when IO failure happens on allocating first page' from Preston Thorpe
closes #2919

Reviewed-by: Pedro Muniz (@pedrocarlo)

Closes #2968
2025-09-08 18:59:34 +03:00
PThorpe92
237b9fefd7 Fix infinite loop when IO failure happens on allocating first page 2025-09-08 11:49:33 -04:00
Pekka Enberg
0c6398c935 core/vdbe: Fix apply_affinity_char() text parsing
We need strict parsing in apply_affinity_char() to avoid transforming
non-numeric values (for example, "1a") into numeric values.
2025-09-08 18:49:13 +03:00
Pekka Enberg
f88f39082a core/vdbe: Fix MakeRecord affinity handling
The MakeRecord instruction now accepts an optional affinity_str
parameter that applies column-specific type conversions before creating
records. When provided, the affinity string is applied
character-by-character to each register using the existing
apply_affinity_char() function, matching SQLite's behavior.

Fixes #2040
Fixes #2041
2025-09-08 18:49:13 +03:00
Pekka Enberg
081a7b563b Merge 'Fix crash in Next opcode if cursor stack has no pages' from Jussi Saurio
Closes #2924
Unsure if this fix is that great, but it does fix the issue described in
#2924 -- added minimal regression test to illustrate the behavior
This crash requires a pretty specific set of circumstances:
- 3-way join with two innermost being left joins
- nullable seek key on the innermost table:
    * middle table gets nulled out because no matches with the outermost
table
    * hence when we seek the innermost table using middle table values,
the seek key is null, so `Insn::IsNull` entirely skips the innermost
table
Perhaps a bytecode plan illustrates this better:
```sql
turso> explain select a.x, b.x, c.x from a left join b on a.y=b.x left join c on b.y=c.x;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     34    0                    0   Start at 34
1     OpenRead           0     2     0                    0   table=a, root=2, iDb=0
2     OpenRead           1     4     0                    0   table=b, root=4, iDb=0
3     OpenRead           2     5     0                    0   index=sqlite_autoindex_b_1, root=5, iDb=0
4     OpenRead           3     7     0                    0   index=sqlite_autoindex_c_1, root=7, iDb=0
5     Rewind             0     33    0                    0   Rewind table a
6       Integer          0     4     0                    0   r[4]=0
7       Column           0     1     6                    0   r[6]=a.y
8       IsNull           6     28    0                    0   if (r[6]==NULL) goto 28
9       SeekGE           2     28    6                    0   key=[6..6]
10        IdxGT          2     28    6                    0   key=[6..6]
11        DeferredSeek   2     1     0                    0   
12        Integer        1     4     0                    0   r[4]=1
13        Integer        0     5     0                    0   r[5]=0
14        Column         1     1     7                    0   r[7]=b.y
-- if b.y is NULL, we skip the entire table loop between insns 16-23
-- except when we call NullRow and then Goto to re-enter that loop in order to
-- return NULL values for the table
15        IsNull         7     24    0                    0   if (r[7]==NULL) goto 24
16        SeekGE         3     24    7                    0   key=[7..7]
17          IdxGT        3     24    7                    0   key=[7..7]
18          Integer      1     5     0                    0   r[5]=1
19          Column       0     0     1                    0   r[1]=a.x
20          Column       1     0     2                    0   r[2]=b.x
21          Column       3     0     3                    0   r[3]=sqlite_autoindex_c_1.x
22          ResultRow    1     3     0                    0   output=r[1..3]
23        Next           3     17    0                    0   
24        IfPos          5     27    0                    0   r[5]>0 -> r[5]-=0, goto 27
25        NullRow        3     0     0                    0   Set cursor 3 to a (pseudo) NULL row
26        Goto           0     18    0                    0   
27      Next             2     10    0                    0   
28      IfPos            4     32    0                    0   r[4]>0 -> r[4]-=0, goto 32
29      NullRow          1     0     0                    0   Set cursor 1 to a (pseudo) NULL row
30      NullRow          2     0     0                    0   Set cursor 2 to a (pseudo) NULL row
31      Goto             0     12    0                    0   
32    Next               0     6     0                    0   
33    Halt               0     0     0                    0   
34    Transaction        0     0     3                    0   iDb=0 write=false
35    Goto               0     1     0                    0
```

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2967
2025-09-08 17:45:29 +03:00
Pekka Enberg
088cb449a5 Merge 'reduce cloning Arc<Page>' from Lâm Hoàng Phúc
only use `self.stack.top()` when we need to store `Arc<Page>` in struct
```sh
Execute `SELECT count() FROM users`/limbo_execute_select_count
                        time:   [5.3733 µs 5.3801 µs 5.3881 µs]
                        change: [-34.047% -33.949% -33.851%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe
```

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2962
2025-09-08 17:02:35 +03:00
Jussi Saurio
5820f691af fix: do not crash in Next if cursor stack has no pages 2025-09-08 16:54:35 +03:00
TcMits
3aa4650f06 make mr.clippy happy 2025-09-08 18:24:50 +07:00
TcMits
a6ff568530 reduce cloning 'Arc<Page>' 2025-09-08 18:00:18 +07:00
Pekka Enberg
01879144b6 Merge 'Evaluate left join seek key condition again after null row' from Jussi Saurio
Closes #2949
This fixes a special case of the behavior described in #2501 - the
special case is that WHERE conditions that were selected as seek
predicates for the left join table were not properly evaluated when the
right-hand-side table returned no match.
The test in commit 12d72d115588a9e744bdb22382998ba1bf9031ab should
demonstrate this adequately - this should return no rows, but on `main`
it returns `1|NULL`.

Closes #2955
2025-09-08 12:01:17 +03:00
Jussi Saurio
c664639c09 Merge 'Add assertion: we read a page with the correct id' from Jussi Saurio
Part of debugging #2746 , but a good sanity check in any case.

Reviewed-by: Avinash Sajjanshetty (@avinassh)

Closes #2802
2025-09-08 09:52:31 +03:00
Jussi Saurio
2c6e48903e Merge 'Prevent setting of encryption keys if already set' from Gaurav Sarma
Fixes https://github.com/tursodatabase/turso/issues/2883
<img width="867" height="128" alt="Screenshot 2025-09-05 at 10 44 18 PM"
src="https://github.com/user-attachments/assets/54a659ba-
cfe1-4622-939b-c7c31362ee5a" />

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Reviewed-by: Avinash Sajjanshetty (@avinassh)

Closes #2914
2025-09-08 09:49:55 +03:00
Jussi Saurio
03cb84ef30 Merge 'expr: use more efficient implementation for binary condition exprs' from Jussi Saurio
Closes #2946
currently we always evaluate the binary expression, then coerce it to
zero/null with the `ZeroOrNull` instruction, and then emit a separate
jump.
this is fine for non-conditional expressions where we are using the
value itself (e.g. in a SELECT result column), but in conditionals we
don't care about that at all and just want to either jump or not jump.
so: try to keep the spirit of code reuse, but still have distinct
implementations for conditionals and non-conditionals.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2947
2025-09-08 09:46:42 +03:00
Jussi Saurio
f6ede79de7 Do not consume non-join WHERE terms in left joins 2025-09-08 09:35:11 +03:00
Nikita Sivukhin
87d49cd039 cargo fmt after rebase 2025-09-07 20:08:10 +04:00
Nikita Sivukhin
cd627c2368 remove unnecessary changes 2025-09-07 19:56:06 +04:00
Nikita Sivukhin
5b9fe0cdf3 fix 2025-09-07 19:56:06 +04:00
Nikita Sivukhin
0b6a6e7713 remove comma 2025-09-07 19:56:06 +04:00
Nikita Sivukhin
9aed831f2f format 2025-09-07 19:56:05 +04:00
Nikita Sivukhin
db7c6b3370 try to speed up count(*) where 1 = 1 2025-09-07 19:55:42 +04:00
Nikita Sivukhin
c374cf0c93 remove Cell/RefCell from PageStack 2025-09-07 19:54:50 +04:00
Preston Thorpe
b7c43cf293 Merge 'Add json_each table-valued function (1-arg only)' from Mikaël Francoeur
This adds the [`json_each`](https://sqlite.org/json1.html#the_json_each_
and_json_tree_table_valued_functions) TVF. Only the 1-arg version is
supported for now.
As suggested in the comments on this PR, I've also extended the virtual
table system to support internal TVF's, as opposed to extensions and
pragma TVF's.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2691
2025-09-06 19:46:20 -04:00
Jussi Saurio
ed3c73a194 kargo klippy 2025-09-06 17:51:15 +03:00
Jussi Saurio
430101ab47 expr: use more efficient implementation for binary condition exprs
currently we always evaluate the binary expression, then coerce it
to zero/null with the `ZeroOrNull` instruction, and then emit a separate
jump.

this is fine for non-conditional expressions where we are using the value
itself (e.g. in a SELECT result column), but in conditionals we don't care
about that at all and just want to jump.

so: try to keep the spirit of code reuse, but still have distinct implementations
for conditionals and non-conditionals.
2025-09-06 17:45:54 +03:00
Gaurav Sarma
b3242a18d9 Prevent setting of encryption keys if already set 2025-09-06 22:37:12 +08:00
Pekka Enberg
9c24b8d088 Merge 'Remove RefCell from Cursor' from Pedro Muniz
Closes #2944
2025-09-06 15:03:23 +03:00
pedrocarlo
e6344db5b1 remove Refcell from Cursor 2025-09-06 01:46:21 -03:00
PThorpe92
01d64977d7 Use more efficient circular list and rely on clock hand for pagecache 2025-09-05 22:40:27 -04:00
PThorpe92
644d0f270b Add evict slot method in page cache 2025-09-05 16:13:30 -04:00
PThorpe92
b89513f031 remove useless saturating sub 2025-09-05 16:13:30 -04:00
PThorpe92
39a47d67e6 Apply PR suggestions 2025-09-05 16:13:29 -04:00
PThorpe92
f45a7538fe Use true sieve/gclock algo instead of lru,dont link pages circilarly 2025-09-05 16:13:29 -04:00
PThorpe92
e418a902e5 Fix scoping issues now that refcells are gone to prevent extra destructors 2025-09-05 16:13:28 -04:00
PThorpe92
c85a61442f Remove type alias in page cache 2025-09-05 16:13:28 -04:00
PThorpe92
5ba273eea5 remove unused impl for refbit 2025-09-05 16:13:28 -04:00
PThorpe92
246b62d513 Remove unnecessary refcells, as PageCacheEntry has interior mutability 2025-09-05 16:13:27 -04:00
PThorpe92
582e25241e Implement GClock algorithm to distinguish between hot pages and scan touches 2025-09-05 16:13:27 -04:00
PThorpe92
254a0a9342 Apply fix and rename ignore_existing to upsert 2025-09-05 16:13:27 -04:00
PThorpe92
3a0b9b360a Fix clippy warnings 2025-09-05 16:13:26 -04:00
PThorpe92
03d5598cfb Use sieve algorithm in page cache in place of full LRU 2025-09-05 16:13:26 -04:00
Mikaël Francoeur
bc08548dc3 put json_each behind feature 2025-09-05 14:56:09 -04:00
Mikaël Francoeur
cf034dbabd fix rebase 2025-09-05 14:56:09 -04:00
Mikaël Francoeur
54719f4eac use new internal virtual table type 2025-09-05 14:56:05 -04:00
Mikaël Francoeur
e6d3d6ea54 1-arg json_each implementation 2025-09-05 14:47:40 -04:00
Jussi Saurio
a0613ef781 Avoid allocating and then immediately fallbacking errors in affinity
On the syscall IO backend, on TPC-H query 12, the _dominating_ part
of the stack trace is trying to construct affinities from a character,
failing, allocating an error&string, and then immediately falling back to
Blob affinity and dropping the error&string.

Since I'm on vacation I won't spend cycles on figuring out why we are passing
an incorrect affinity in `flags.get_affinity()` and instead make this lazy PR
just to improve performance and stop doing silly things :]
2025-09-05 18:34:23 +03:00
Pere Diaz Bou
4ddf9c23de core/pager: assert-ready-page-sanity fmt for jussi 2025-09-05 16:52:33 +02:00
Pere Diaz Bou
382a1e14ca Merge 'core: handle edge cases for read_varint' from Sonny
Add handling malformed inputs to function `read_varint` and test cases.
```
# 9 byte truncated to 8
read_varint(&[0x81, 0x80, 0x80, 0x80, 0x80, 0x80, 0x80, 0x80]) 
    before -> panic index out of bounds: the len is 8 but the index is 8
    after -> LimboError
    
# bits set without end
read_varint(&[0x80; 9]) 
    before -> Ok((128, 9))
    after -> LimboError
```

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #2904
2025-09-05 16:15:42 +02:00
Pekka Enberg
811c5a7ce0 Merge 'Fix float formatting and comparison + Blob concat' from Levy A.
These changes can be verified with the expression fuzzer. Fixes
https://github.com/tursodatabase/turso/issues/2881.
- Compatible float formatting
- Compatible integer-float comparisons
- Blob concatenation

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #2929
2025-09-05 17:02:51 +03:00
Pekka Enberg
b2664e12c2 cargo fmt 2025-09-05 16:12:12 +03:00