Commit Graph

10673 Commits

Author SHA1 Message Date
Nikita Sivukhin
8b1b71d8b0 fix clippy 2025-09-09 12:21:54 +04:00
Nikita Sivukhin
9f891f891c adjust github workflow 2025-09-09 12:21:29 +04:00
Nikita Sivukhin
816aa8b2bc small fixes 2025-09-09 12:05:41 +04:00
Nikita Sivukhin
8160f4dc04 restructure js bindings 2025-09-09 11:32:38 +04:00
Nikita Sivukhin
96a595069c add browser-specific OPFS file system and few other helpers 2025-09-09 11:32:38 +04:00
Nikita Sivukhin
4d80f8255f use MemoryIO for ephemeral tables for wasm target 2025-09-09 11:20:24 +04:00
Nikita Sivukhin
794440606a sligthly adjust attach to use same IO as main DB (if no custom VFS is specified) 2025-09-09 11:07:26 +04:00
Pekka Enberg
457aaeb1a7 Merge 'optimizer: convert outer join to inner join if possible' from Jussi Saurio
Convert `LEFT JOIN` to `INNER JOIN` when the result of `LEFT JOIN` can
never be different from the result of an `INNER JOIN`
This is useful because 1. it uses less instructions and 2. it allows for
join reordering due to inner join commutativity

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2972
2025-09-09 08:50:31 +03:00
Pekka Enberg
0ef857f98e Merge 'Handle case where null flag is set in op_column' from Jussi Saurio
current code is incorrectly assuming that `index_cursor.rowid()` always
finds a rowid, but this is not the case when `NullRow` has previously
been called.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2973
2025-09-09 08:49:22 +03:00
Pekka Enberg
c7c669b5af Merge 'Fix merge script to prompt if tests are still in progress' from Preston Thorpe
Closes #2969
2025-09-09 08:49:08 +03:00
Pekka Enberg
b911a87070 Merge 'SQL generation fixes' from Pekka Enberg
Closes #2956
2025-09-09 08:48:51 +03:00
Pekka Enberg
3787126aea Merge 'remove &1' from Lâm Hoàng Phúc
Closes #2976
2025-09-09 08:48:41 +03:00
TcMits
5167096893 remove &1 2025-09-09 11:52:16 +07:00
Jussi Saurio
b01033975a regression test for crash in op_column 2025-09-09 00:11:19 +03:00
Jussi Saurio
c930f28643 Handle case where null flag is set in op_column 2025-09-09 00:00:19 +03:00
Jussi Saurio
2d1d284279 optimizer: convert outer join to inner join if possible 2025-09-08 23:21:34 +03:00
PThorpe92
d2a762534c Fix merge script to prompt if tests are still in progress 2025-09-08 13:22:40 -04:00
PThorpe92
ccae3ab0f2 Change callsites to cancel any further IO when an error occurs and drain 2025-09-08 13:18:40 -04:00
PThorpe92
a750505762 Impl cancel and drain methods for io_uring 2025-09-08 13:18:38 -04:00
PThorpe92
eb0e069445 Add ShortWrite to CompletionError 2025-09-08 13:18:09 -04:00
PThorpe92
02df372811 Add cancel and drain methods to IO trait 2025-09-08 13:18:03 -04:00
Preston Thorpe
467b925256 Merge 'add gen-bigass-database.py' from Jussi Saurio
#2928 was reverted, but let's at least add the ability to create an
equivalent big schema so that the issues reported by Preston using that
larger schema can be reproduced and fixed.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2960
2025-09-08 13:06:15 -04:00
Jussi Saurio
cc46bfb6d8 add gen-bigass-database.py
ability to create an equivalent big schema so that the issues
reported by Preston using that larger schema can be reproduced
and fixed.
2025-09-08 13:05:33 -04:00
Preston Thorpe
8d05336522 Merge 'Fix affinity handling in MakeRecord' from Pekka Enberg
Closes #2966
2025-09-08 12:14:33 -04:00
Pekka Enberg
71a812ce55 Merge 'Fix infinite loop when IO failure happens on allocating first page' from Preston Thorpe
closes #2919

Reviewed-by: Pedro Muniz (@pedrocarlo)

Closes #2968
2025-09-08 18:59:34 +03:00
Pekka Enberg
86baa06600 tests/integration: Add affinity differential fuzz test 2025-09-08 18:59:20 +03:00
PThorpe92
237b9fefd7 Fix infinite loop when IO failure happens on allocating first page 2025-09-08 11:49:33 -04:00
Pekka Enberg
0c6398c935 core/vdbe: Fix apply_affinity_char() text parsing
We need strict parsing in apply_affinity_char() to avoid transforming
non-numeric values (for example, "1a") into numeric values.
2025-09-08 18:49:13 +03:00
Pekka Enberg
7f002548fd tests/integration: Enable min_max_agg_fuzz() test case 2025-09-08 18:49:13 +03:00
Pekka Enberg
f88f39082a core/vdbe: Fix MakeRecord affinity handling
The MakeRecord instruction now accepts an optional affinity_str
parameter that applies column-specific type conversions before creating
records. When provided, the affinity string is applied
character-by-character to each register using the existing
apply_affinity_char() function, matching SQLite's behavior.

Fixes #2040
Fixes #2041
2025-09-08 18:49:13 +03:00
Mikaël Francoeur
b480b526bc implement 2-args json_each 2025-09-08 11:34:17 -04:00
Pekka Enberg
081a7b563b Merge 'Fix crash in Next opcode if cursor stack has no pages' from Jussi Saurio
Closes #2924
Unsure if this fix is that great, but it does fix the issue described in
#2924 -- added minimal regression test to illustrate the behavior
This crash requires a pretty specific set of circumstances:
- 3-way join with two innermost being left joins
- nullable seek key on the innermost table:
    * middle table gets nulled out because no matches with the outermost
table
    * hence when we seek the innermost table using middle table values,
the seek key is null, so `Insn::IsNull` entirely skips the innermost
table
Perhaps a bytecode plan illustrates this better:
```sql
turso> explain select a.x, b.x, c.x from a left join b on a.y=b.x left join c on b.y=c.x;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     34    0                    0   Start at 34
1     OpenRead           0     2     0                    0   table=a, root=2, iDb=0
2     OpenRead           1     4     0                    0   table=b, root=4, iDb=0
3     OpenRead           2     5     0                    0   index=sqlite_autoindex_b_1, root=5, iDb=0
4     OpenRead           3     7     0                    0   index=sqlite_autoindex_c_1, root=7, iDb=0
5     Rewind             0     33    0                    0   Rewind table a
6       Integer          0     4     0                    0   r[4]=0
7       Column           0     1     6                    0   r[6]=a.y
8       IsNull           6     28    0                    0   if (r[6]==NULL) goto 28
9       SeekGE           2     28    6                    0   key=[6..6]
10        IdxGT          2     28    6                    0   key=[6..6]
11        DeferredSeek   2     1     0                    0   
12        Integer        1     4     0                    0   r[4]=1
13        Integer        0     5     0                    0   r[5]=0
14        Column         1     1     7                    0   r[7]=b.y
-- if b.y is NULL, we skip the entire table loop between insns 16-23
-- except when we call NullRow and then Goto to re-enter that loop in order to
-- return NULL values for the table
15        IsNull         7     24    0                    0   if (r[7]==NULL) goto 24
16        SeekGE         3     24    7                    0   key=[7..7]
17          IdxGT        3     24    7                    0   key=[7..7]
18          Integer      1     5     0                    0   r[5]=1
19          Column       0     0     1                    0   r[1]=a.x
20          Column       1     0     2                    0   r[2]=b.x
21          Column       3     0     3                    0   r[3]=sqlite_autoindex_c_1.x
22          ResultRow    1     3     0                    0   output=r[1..3]
23        Next           3     17    0                    0   
24        IfPos          5     27    0                    0   r[5]>0 -> r[5]-=0, goto 27
25        NullRow        3     0     0                    0   Set cursor 3 to a (pseudo) NULL row
26        Goto           0     18    0                    0   
27      Next             2     10    0                    0   
28      IfPos            4     32    0                    0   r[4]>0 -> r[4]-=0, goto 32
29      NullRow          1     0     0                    0   Set cursor 1 to a (pseudo) NULL row
30      NullRow          2     0     0                    0   Set cursor 2 to a (pseudo) NULL row
31      Goto             0     12    0                    0   
32    Next               0     6     0                    0   
33    Halt               0     0     0                    0   
34    Transaction        0     0     3                    0   iDb=0 write=false
35    Goto               0     1     0                    0
```

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2967
2025-09-08 17:45:29 +03:00
Pekka Enberg
088cb449a5 Merge 'reduce cloning Arc<Page>' from Lâm Hoàng Phúc
only use `self.stack.top()` when we need to store `Arc<Page>` in struct
```sh
Execute `SELECT count() FROM users`/limbo_execute_select_count
                        time:   [5.3733 µs 5.3801 µs 5.3881 µs]
                        change: [-34.047% -33.949% -33.851%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe
```

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2962
2025-09-08 17:02:35 +03:00
Pekka Enberg
d05ea4fd70 Merge 'cli: Fix dump compatibility in "PRAGMA foreign_keys"' from Pekka Enberg
SQLite emits a semicolon after "PRAGMA foreign_keys=OFF" so let's do th
same.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2961
2025-09-08 16:56:01 +03:00
Jussi Saurio
5820f691af fix: do not crash in Next if cursor stack has no pages 2025-09-08 16:54:35 +03:00
Diego Reis
e94f18610d workflow: Minor perf CI adjustments
- Use github's default runner due blacksmith's higher performance
variance - Always comment on PRs about perf variances (currently only
for people with write permissions)
2025-09-08 09:23:25 -03:00
TcMits
3aa4650f06 make mr.clippy happy 2025-09-08 18:24:50 +07:00
Pekka Enberg
a2a5b852b8 Document query result ordering limitation 2025-09-08 14:19:48 +03:00
TcMits
a6ff568530 reduce cloning 'Arc<Page>' 2025-09-08 18:00:18 +07:00
Pavan-Nambi
d757a330ee use sqlite_int_float_compare 2025-09-08 16:26:37 +05:30
Pekka Enberg
80a4358272 cli: Fix dump compatibility in "PRAGMA foreign_keys"
SQLite emits a semicolon after "PRAGMA foreign_keys=OFF" so let's do th
same.
2025-09-08 12:55:29 +03:00
Pekka Enberg
01879144b6 Merge 'Evaluate left join seek key condition again after null row' from Jussi Saurio
Closes #2949
This fixes a special case of the behavior described in #2501 - the
special case is that WHERE conditions that were selected as seek
predicates for the left join table were not properly evaluated when the
right-hand-side table returned no match.
The test in commit 12d72d115588a9e744bdb22382998ba1bf9031ab should
demonstrate this adequately - this should return no rows, but on `main`
it returns `1|NULL`.

Closes #2955
2025-09-08 12:01:17 +03:00
Pekka Enberg
93baef6ae4 Merge 'use mlugg/setup-zig instead of unmaintained action' from Kingsword
link: https://github.com/goto-bus-stop/setup-zig?tab=readme-ov-
file#setup-zig

Closes #2951
2025-09-08 12:00:57 +03:00
Pekka Enberg
c8d034ed04 sql_generation: Fix WHERE clause generation
We're currently only generating `WHERE (TRUE)` and `WHERE (FALSE)`. Fix that.
2025-09-08 11:59:45 +03:00
Pekka Enberg
9f6c11a74f sql_generation: Fix predicate column indexing
The number of columns in the row can be less than the number of columns in the
table so fix out of bounds error in indexing.
2025-09-08 11:59:45 +03:00
Jussi Saurio
c664639c09 Merge 'Add assertion: we read a page with the correct id' from Jussi Saurio
Part of debugging #2746 , but a good sanity check in any case.

Reviewed-by: Avinash Sajjanshetty (@avinassh)

Closes #2802
2025-09-08 09:52:31 +03:00
Jussi Saurio
2c6e48903e Merge 'Prevent setting of encryption keys if already set' from Gaurav Sarma
Fixes https://github.com/tursodatabase/turso/issues/2883
<img width="867" height="128" alt="Screenshot 2025-09-05 at 10 44 18 PM"
src="https://github.com/user-attachments/assets/54a659ba-
cfe1-4622-939b-c7c31362ee5a" />

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Reviewed-by: Avinash Sajjanshetty (@avinassh)

Closes #2914
2025-09-08 09:49:55 +03:00
Jussi Saurio
b99a4f528d Merge 'support float without fractional part' from Lâm Hoàng Phúc
fix #2933

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2935
2025-09-08 09:49:20 +03:00
Jussi Saurio
03cb84ef30 Merge 'expr: use more efficient implementation for binary condition exprs' from Jussi Saurio
Closes #2946
currently we always evaluate the binary expression, then coerce it to
zero/null with the `ZeroOrNull` instruction, and then emit a separate
jump.
this is fine for non-conditional expressions where we are using the
value itself (e.g. in a SELECT result column), but in conditionals we
don't care about that at all and just want to either jump or not jump.
so: try to keep the spirit of code reuse, but still have distinct
implementations for conditionals and non-conditionals.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2947
2025-09-08 09:46:42 +03:00
Jussi Saurio
457f1980ca Merge 'Small fixes' from Nikita Sivukhin
Attempt to improve performance of `SELECT COUNT(*) FROM t WHERE 1 = 1`
(redundant filter added to force sqlite and tursodb to iterate over rows
instead of applying count optimization).
Before that PR turso with hot cache runs for ~440ms.
After that PR turso with hot cache runs for ~200ms.
This PR eliminates small things on a hot path and also introduce
separate flow for `get_next_record` which just increment cell idx if its
safe to do that - without much additional overhead.
Also, this PR eliminates RefCell/Cell from PageStack because there were
completely unnecessary.

Reviewed-by: Preston Thorpe <preston@turso.tech>
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #2887
2025-09-08 09:45:35 +03:00