aljaz/turso
1
0
Fork 0
mirror of https://github.com/aljazceru/turso.git synced 2026-01-02 07:54:19 +01:00
Code Issues Packages Projects Releases Wiki Activity
8,050 Commits 1 Branch 0 Tags
f7ad55b6809e9f04d2db1335cf95e536e970239a
Commit Graph

8050 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Nikita Sivukhin
f7ad55b680 remove unnecessary argument 2025-08-25 12:24:39 +04:00
Pekka Enberg
f452f54ce3 Merge 'sqlite3: Implement sqlite3_db_filename()' from Pekka Enberg 
Closes #2771
 2025-08-25 11:23:13 +03:00
Pekka Enberg
262ead8240 Merge 'ANALYZE creates sqlite_stat1 if it doesn't exist' from Alex Miller 
This is change 2/N of #656
This change replaces a bail_parse_error!() when sqlite_stat1 doesn't
exist with the appropriate codegen to create the table, and handle both
cases of the table existing or not existing.
SQLite's codegen looks like:
```
sqlite> create table stat_test(a,b,c);
sqlite> explain analyze stat_test;
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     40    0                    0   Start at 40
1     ReadCookie     0     3     2                    0
2     If             3     5     0                    0
3     SetCookie      0     2     4                    0
4     SetCookie      0     5     1                    0
5     CreateBtree    0     2     1                    0   r[2]=root iDb=0 flags=1
6     OpenWrite      0     1     0     5              0   root=1 iDb=0
7     NewRowid       0     1     0                    0   r[1]=rowid
8     Blob           6     3     0                   0   r[3]= (len=6)
9     Insert         0     3     1                    8   intkey=r[1] data=r[3]
10    Close          0     0     0                    0
11    Close          0     0     0                    0
12    Null           0     4     5                    0   r[4..5]=NULL
13    Noop           4     0     4                    0
14    OpenWrite      3     1     0     5              0   root=1 iDb=0; sqlite_master
15    SeekRowid      3     17    1                    0   intkey=r[1]
16    Rowid          3     5     0                    0   r[5]= rowid of 3
17    IsNull         5     26    0                    0   if r[5]==NULL goto 26
18    String8        0     6     0     table          0   r[6]='table'
19    String8        0     7     0     sqlite_stat1   0   r[7]='sqlite_stat1'
20    String8        0     8     0     sqlite_stat1   0   r[8]='sqlite_stat1'
21    Copy           2     9     0                    0   r[9]=r[2]
22    String8        0     10    0     CREATE TABLE sqlite_stat1(tbl,idx,stat) 0   r[10]='CREATE TABLE sqlite_stat1(tbl,idx,stat)'
23    MakeRecord     6     5     4     BBBDB          0   r[4]=mkrec(r[6..10])
24    Delete         3     68    5                    0
25    Insert         3     4     5                    0   intkey=r[5] data=r[4]
26    SetCookie      0     1     2                    0
27    ParseSchema    0     0     0     tbl_name='sqlite_stat1' AND type!='trigger' 0
28    OpenWrite      0     2     0     3              16  root=2 iDb=0; sqlite_stat1
29    OpenRead       5     2     0     3              0   root=2 iDb=0; stat_test
30    String8        0     18    0     stat_test      0   r[18]='stat_test'; stat_test
31    Count          5     20    0                    0   r[20]=count()
32    IfNot          20    37    0                    0
33    Null           0     19    0                    0   r[19]=NULL
34    MakeRecord     18    3     16    BBB            0   r[16]=mkrec(r[18..20])
35    NewRowid       0     12    0                    0   r[12]=rowid
36    Insert         0     16    12                   8   intkey=r[12] data=r[16]
37    LoadAnalysis   0     0     0                    0
38    Expire         0     0     0                    0
39    Halt           0     0     0                    0
40    Transaction    0     1     1     0              1   usesStmtJournal=1
41    Goto           0     1     0                    0
```
And now Turso's looks like:
```
turso> create table stat_test(a,b,c);
turso> explain analyze stat_test;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     23    0                    0   Start at 23
1     Null               0     1     0                    0   r[1]=NULL
2     CreateBtree        0     2     1                    0   r[2]=root iDb=0 flags=1
3     OpenWrite          0     1     0                    0   root=1; iDb=0
4     NewRowid           0     3     0                    0   r[3]=rowid
5     String8            0     4     0     table          0   r[4]='table'
6     String8            0     5     0     sqlite_stat1   0   r[5]='sqlite_stat1'
7     String8            0     6     0     sqlite_stat1   0   r[6]='sqlite_stat1'
8     Copy               2     7     0                    0   r[7]=r[2]
9     String8            0     8     0     CREATE TABLE sqlite_stat1(tbl,idx,stat)  0   r[8]='CREATE TABLE sqlite_stat1(tbl,idx,stat)'
10    MakeRecord         4     5     9                    0   r[9]=mkrec(r[4..8])
11    Insert             0     9     3     sqlite_stat1   0   intkey=r[3] data=r[9]
12    ParseSchema        0     0     0     tbl_name = 'sqlite_stat1' AND type != 'trigger'  0   tbl_name = 'sqlite_stat1' AND type != 'trigger'
13    OpenWrite          1     2     0                    0   root=2; iDb=0
14    OpenRead           2     2     0                    0   =stat_test, root=2, iDb=0
15    String8            0     12    0     stat_test      0   r[12]='stat_test'
16    Count              2     14    0                    0
17    IfNot              14    22    0                    0   if !r[14] goto 22
18    Null               0     13    0                    0   r[13]=NULL
19    MakeRecord         12    3     11                   0   r[11]=mkrec(r[12..14])
20    NewRowid           1     10    0                    0   r[10]=rowid
21    Insert             1     11    10    sqlite_stat1   0   intkey=r[10] data=r[11]
22    Halt               0     0     0                    0
23    Goto               0     1     0                    0
```
The notable difference in size is following the same codegen difference
in CREATE TABLE, where sqlite's odd dance of adding a placeholder entry
which is immediately replaced is instead done in tursodb as just
inserting the correct row in the first place. Aside from lines 6-13 of
sqlite's vdbe being missing, there's still the lack of LoadAnalysis,
Expire, and Cookie management.

Closes #2765
 2025-08-25 10:24:46 +03:00
Pekka Enberg
b162f89b73 sqlite3: Implement sqlite3_db_filename() 2025-08-25 10:20:50 +03:00
Pekka Enberg
4bca5edb9e Merge 'Don't clear transaction state in nested statement' from Jussi Saurio 
Closes #2738
Closes #2739
Closes #2753
Closes #2755
Closes #2767
Closes #2768
Closes #2769
Closes #2770
## El problema
If a connection does e.g. CREATE TABLE, it will start a "child
statement" to reparse the schema. That statement does not start its own
transaction, and so should not try to end the existing one either.
We had a logic bug where these steps would happen:
- `CREATE TABLE` executed successfully
- pread fault happens inside `ParseSchema` child stmt
- `handle_program_error()` is called
- `pager.end_tx()` returns immediately because `is_nested_stmt` is true
and we correctly no-op it.
- however, crucially: `handle_program_error()` then sets tx state to
None
- parent statement now catches error from nested stmt and calls
`handle_program_error()`, which calls `pager.end_tx()` again, and since
txn state is None, when it calls `rollback()` we panic on the assertion
`"dirty pages should be empty for read txn"`
## La solucion
Do not do _any_ error processing in `handle_program_error()` inside a
nested stmt. This means that the parent write txn is still active when
it processes the error from the child and we avoid this panic.

Closes #2772
 2025-08-25 10:12:09 +03:00
Jussi Saurio
1777440af7 Merge 'refactor/btree: rewrite the free_cell_range() function' from Jussi Saurio 
i had a rough time reading this function earlier and trying to
understand it, so rewrote it in a way that, to me, is much more
readable.

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #2509
 2025-08-25 10:08:11 +03:00
Jussi Saurio
4ea8cd0007 refactor/btree: rewrite the free_cell_range() function 
i had a rough time reading this function earlier and trying to understand it,
so rewrote it in a way that, to me, is much more readable.
 2025-08-25 09:41:44 +03:00
Jussi Saurio
aa07b81e48 Merge 'flake.nix: add uv dependency to nativeBuildInputs' from Ceferino Patino 
`make test` fails when using the nix-shell environment due to `uv` not
being included in the list of dependencies with
`make: uv: No such file or directory
make: *** [Makefile:55: uv-sync-test] Error 127`
simple fix adding it to `nativeBuildInputs` of the shell. after that
runs as expected.

Reviewed-by: Levy A. (@levydsa)

Closes #2636
 2025-08-25 09:18:15 +03:00
Jussi Saurio
f2598a2dea Merge 'Remove Result from signature' from Mikaël Francoeur 
This PR removes `Result<()>` from `Jsonb::write_to_string()`, since it
wasn't required. The method now returns `()`.

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #2717
 2025-08-25 09:17:37 +03:00
Jussi Saurio
54ff656c9d Do not clear txn state inside nested statement 
If a connection does e.g. CREATE TABLE, it will start a "child statement"
to reparse the schema. That statement does not start its own transaction,
and so should not try to end the existing one either.

We had a logic bug where these steps would happen:

- `CREATE TABLE` executed successfully
- pread fault happens inside `ParseSchema` child stmt
- `handle_program_error()` is called
- `pager.end_tx()` returns immediately because `is_nested_stmt` is true
  and we correctly no-op it.
- however, crucially: `handle_program_error()` then sets tx state to None
- parent statement now catches error from nested stmt and calls
  `handle_program_error()`, which calls `pager.end_tx()` again, and since
  txn state is None, when it calls `rollback()` we panic on the assertion
 `"dirty pages should be empty for read txn"`

Solution:

Do not do _any_ error processing in `handle_program_error()` inside a nested
stmt. This means that the parent write txn is still active when it processes
the error from the child and we avoid this panic.
 2025-08-25 08:49:22 +03:00
Pekka Enberg
2e11e1bb8c Merge 'Switch to F_FULLSYNC on Darwin' from 
Closes #2743 .

Closes #2766
 2025-08-25 08:09:29 +03:00
rajajisai
9068a29380 Use unsafe block 2025-08-24 18:56:05 -04:00
rajajisai
84d20ba60f Use F_FULLSYNC in darwin based operating systems 2025-08-24 18:45:46 -04:00
Alex Miller
370da9fa59 ANALYZE creates sqlite_stat1 if it doesn't exist 
This change replaces a bail_parse_error!() when sqlite_stat1 doesn't
exist with the appropriate codegen to create the table, and handle both
cases of the table existing or not existing.

SQLite's codegen looks like:

sqlite> create table stat_test(a,b,c);
sqlite> explain analyze stat_test;
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     40    0                    0   Start at 40
1     ReadCookie     0     3     2                    0
2     If             3     5     0                    0
3     SetCookie      0     2     4                    0
4     SetCookie      0     5     1                    0
5     CreateBtree    0     2     1                    0   r[2]=root iDb=0 flags=1
6     OpenWrite      0     1     0     5              0   root=1 iDb=0
7     NewRowid       0     1     0                    0   r[1]=rowid
8     Blob           6     3     0                   0   r[3]= (len=6)
9     Insert         0     3     1                    8   intkey=r[1] data=r[3]
10    Close          0     0     0                    0
11    Close          0     0     0                    0
12    Null           0     4     5                    0   r[4..5]=NULL
13    Noop           4     0     4                    0
14    OpenWrite      3     1     0     5              0   root=1 iDb=0; sqlite_master
15    SeekRowid      3     17    1                    0   intkey=r[1]
16    Rowid          3     5     0                    0   r[5]= rowid of 3
17    IsNull         5     26    0                    0   if r[5]==NULL goto 26
18    String8        0     6     0     table          0   r[6]='table'
19    String8        0     7     0     sqlite_stat1   0   r[7]='sqlite_stat1'
20    String8        0     8     0     sqlite_stat1   0   r[8]='sqlite_stat1'
21    Copy           2     9     0                    0   r[9]=r[2]
22    String8        0     10    0     CREATE TABLE sqlite_stat1(tbl,idx,stat) 0   r[10]='CREATE TABLE sqlite_stat1(tbl,idx,stat)'
23    MakeRecord     6     5     4     BBBDB          0   r[4]=mkrec(r[6..10])
24    Delete         3     68    5                    0
25    Insert         3     4     5                    0   intkey=r[5] data=r[4]
26    SetCookie      0     1     2                    0
27    ParseSchema    0     0     0     tbl_name='sqlite_stat1' AND type!='trigger' 0
28    OpenWrite      0     2     0     3              16  root=2 iDb=0; sqlite_stat1
29    OpenRead       5     2     0     3              0   root=2 iDb=0; stat_test
30    String8        0     18    0     stat_test      0   r[18]='stat_test'; stat_test
31    Count          5     20    0                    0   r[20]=count()
32    IfNot          20    37    0                    0
33    Null           0     19    0                    0   r[19]=NULL
34    MakeRecord     18    3     16    BBB            0   r[16]=mkrec(r[18..20])
35    NewRowid       0     12    0                    0   r[12]=rowid
36    Insert         0     16    12                   8   intkey=r[12] data=r[16]
37    LoadAnalysis   0     0     0                    0
38    Expire         0     0     0                    0
39    Halt           0     0     0                    0
40    Transaction    0     1     1     0              1   usesStmtJournal=1
41    Goto           0     1     0                    0

And now Turso's looks like:

turso> create table stat_test(a,b,c);
turso> explain analyze stat_test;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     23    0                    0   Start at 23
1     Null               0     1     0                    0   r[1]=NULL
2     CreateBtree        0     2     1                    0   r[2]=root iDb=0 flags=1
3     OpenWrite          0     1     0                    0   root=1; iDb=0
4     NewRowid           0     3     0                    0   r[3]=rowid
5     String8            0     4     0     table          0   r[4]='table'
6     String8            0     5     0     sqlite_stat1   0   r[5]='sqlite_stat1'
7     String8            0     6     0     sqlite_stat1   0   r[6]='sqlite_stat1'
8     Copy               2     7     0                    0   r[7]=r[2]
9     String8            0     8     0     CREATE TABLE sqlite_stat1(tbl,idx,stat)  0   r[8]='CREATE TABLE sqlite_stat1(tbl,idx,stat)'
10    MakeRecord         4     5     9                    0   r[9]=mkrec(r[4..8])
11    Insert             0     9     3     sqlite_stat1   0   intkey=r[3] data=r[9]
12    ParseSchema        0     0     0     tbl_name = 'sqlite_stat1' AND type != 'trigger'  0   tbl_name = 'sqlite_stat1' AND type != 'trigger'
13    OpenWrite          1     2     0                    0   root=2; iDb=0
14    OpenRead           2     2     0                    0   =stat_test, root=2, iDb=0
15    String8            0     12    0     stat_test      0   r[12]='stat_test'
16    Count              2     14    0                    0
17    IfNot              14    22    0                    0   if !r[14] goto 22
18    Null               0     13    0                    0   r[13]=NULL
19    MakeRecord         12    3     11                   0   r[11]=mkrec(r[12..14])
20    NewRowid           1     10    0                    0   r[10]=rowid
21    Insert             1     11    10    sqlite_stat1   0   intkey=r[10] data=r[11]
22    Halt               0     0     0                    0
23    Goto               0     1     0                    0

The notable difference in size is following the same codegen difference
in CREATE TABLE, where sqlite's odd dance of adding a placeholder entry
which is immediately replaced is instead done in tursodb as just
inserting the correct row in the first place. Aside from lines 6-13 of
sqlite's vdbe being missing, there's still the lack of LoadAnalysis,
Expire, and Cookie management.
 2025-08-24 13:35:39 -07:00
Pekka Enberg
de16c770c3 Merge 'Remove duplicated attribute in ' from bit-aloo 
Removed the redundant `#![allow(clippy::arc_with_non_send_sync)]` from
wal.rs, since it’s already defined in `mod.rs`.

Closes #2764
 2025-08-24 20:54:26 +03:00
Pekka Enberg
7291b59418 Merge 'sqlite3: Implement sqlite3_bind_parameter_index()' from Pekka Enberg 
Closes #2763
 2025-08-24 20:54:18 +03:00
bit-aloo
37cebb0669 fix(clippy): remove duplicate arc_with_non_send_sync attribute in wal.rs 2025-08-24 22:59:47 +05:30
Pekka Enberg
c428ff06b2 sqlite3: Implement sqlite3_bind_parameter_index() 2025-08-24 20:10:31 +03:00
Pekka Enberg
4fc7b94a6b Merge 'sqlite3: Implement sqlite3_clear_bindings()' from Pekka Enberg 
Closes #2759
 2025-08-24 19:57:14 +03:00
Pekka Enberg
9d2f26bb04 sqlite3: Implement sqlite3_clear_bindings() 2025-08-24 19:33:18 +03:00
Pekka Enberg
5a8f281555 Merge 'sqlite3: Implement sqlite3_get_autocommit()' from Pekka Enberg 
Closes #2758
 2025-08-24 16:53:04 +03:00
Pekka Enberg
a8ab049a36 Merge 'Add support for AEGIS encryption algorithm' from Avinash Sajjanshetty 
Depends on #2722
This builds upon the #2722 PR which let me configure the encryption
algorithm. So this adds AEGIS and uses it as default.
Note that choice of cipher at higher APIs is still not possible. I have
a follow up PR which updates the PRAGMAs
AEGIS is way too damn fast, here are some numbers:
```
* MACs:

aegis128x4-mac      : 223.91 Gb/s
aegis128x2-mac      : 270.87 Gb/s
aegis128l-mac       : 229.35 Gb/s
sthash              : 83.60 Gb/s
hmac-sha256 (boring): 27.46 Gb/s
blake3              : 21.41 Gb/s

* Encryption:

aegis128x4          : 104.19 Gb/s
aegis128x2          : 182.46 Gb/s
aegis128l           : 181.62 Gb/s
aegis256x2          : 133.45 Gb/s
aegis256x4          : 125.23 Gb/s
aegis256            : 102.12 Gb/s
aes128-gcm (aes-gcm): 2.16 Gb/s
aes128-gcm (boring) : 63.25 Gb/s
aes256-gcm (aes-gcm): 1.70 Gb/s
aes256-gcm (boring) : 59.14 Gb/s
chacha20-poly1305   : 2.39 Gb/s
ascon128a           : 5.84 Gb/s
```

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2742
 2025-08-24 14:16:10 +03:00
Pekka Enberg
f5dfbfffc2 Merge branch 'main' into aegis 2025-08-24 14:16:01 +03:00
Pekka Enberg
ea2192c332 sqlite3: Implement sqlite3_get_autocommit() 2025-08-24 14:13:36 +03:00
Pekka Enberg
2c6fa76437 cargo fmt 2025-08-24 14:13:20 +03:00
Avinash Sajjanshetty
011f878158 make clippy bro happy 2025-08-24 16:21:06 +05:30
Avinash Sajjanshetty
77a4e96022 run encryption tests in CI 2025-08-24 16:15:13 +05:30
Avinash Sajjanshetty
a4b9c33b81 Use the new API to init cipher 2025-08-24 16:15:13 +05:30
Avinash Sajjanshetty
53f9c0dc7a Add support for lord AEGIS, the fastest and the greatest 2025-08-24 16:15:11 +05:30
Pekka Enberg
73414663f8 Merge 'bindings/java: Implement batch operations for JDBC4Statement' from Kim Seon Woo 
Implement the following methods for JDBC4Statement:
- addBatch
- clearBatch
- executeBatch

Closes #2754
 2025-08-24 08:54:44 +03:00
Pekka Enberg
147740c95e Merge 'reduce cloning Token in parser' from Lâm Hoàng Phúc 
small changes make parser 3-6% faster

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2745
 2025-08-24 08:53:35 +03:00
Pekka Enberg
22c9cb6618 s/PerConnEncryptionContext/EncryptionContext/ 2025-08-24 08:17:20 +03:00
Pekka Enberg
1b89273f10 Merge 'refactor encryption module and make it configurable' from Avinash Sajjanshetty 
Previously, the encryption module had hardcoded a lot of things. This
refactor makes it slightly nice and makes it configurable.
Right now cipher algorithm is assumed and hardcoded, I will make that
configurable in the upcoming PR

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2722
 2025-08-24 08:16:28 +03:00
김선우
7057c97cfe Remove .rustc_info.json 2025-08-24 10:25:14 +09:00
김선우
fa8896d9ee Nit 2025-08-24 10:20:39 +09:00
김선우
9f6eb8bc92 Update verification of batch compatible statements using regex 2025-08-24 10:13:04 +09:00
김선우
bf1473dc08 Override JDBC4PreparedStatement to throw exception when calling addBatch method 2025-08-24 09:35:29 +09:00
김선우
346525e5f0 Update test 2025-08-24 09:25:59 +09:00
김선우
df41994ecc Implement execute batch 2025-08-24 09:15:07 +09:00
Preston Thorpe
b113c497aa Merge 'Replace a couple refcells for types that trivially impl Copy' from Preston Thorpe 
Closes #2750
 2025-08-23 16:33:07 -04:00
Preston Thorpe
dccd09049e Merge 'fix: normalize quotes in update' from 
fixes: #2744
```
turso> create table simple(x);
turso> insert into "simple"(x) values (1);
turso> select * from "simple";
┌───┐
│ x │
├───┤
│ 1 │
└───┘
turso> update "simple" set x = 3 where "simple"."x" = 1;
turso> select * from "simple";
┌───┐
│ x │
├───┤
│ 3 │
└───┘
turso>
```

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2747
 2025-08-23 16:31:44 -04:00
PThorpe92
9a418f1d3e Replace a couple refcells with cell in pager 2025-08-23 15:55:01 -04:00
themixednuts
80eca66be9 fix: normalize quotes in update 
fixes: #2744
 2025-08-23 03:17:03 -05:00
TcMits
fd63688ede reduce cloning Token in parser 2025-08-23 15:07:32 +07:00
Pekka Enberg
52f66e6c60 Merge 'Add syntax highlighting for EXPLAIN and ANALYZE' from Alex Miller 
I'm working on ANALYZE.  I'm using EXPLAIN.  The lack of highlighting
for them in the CLI annoyed me a bit.
I don't think there's any tests for this?  I'm mostly at a "it seems to
work for me".  I double checked that `EXPLAIN SELECT CASE 0 WHEN 0 THEN
0 ELSE 1` syntax highlights, to make sure  I didn't break the longer
parsing (which I had).

Closes #2741
 2025-08-23 10:51:31 +03:00
Pekka Enberg
dd56eb7d9b Merge 'Add basic support for ANALYZE statement' from Alex Miller 
This is part 1 of N implementing #656
This permits only `ANALYZE <table_name>` to work, and all other forms
fail with a parse error (as documented in the tests).
On SQLite, ANALYZE generates:
```
sqlite> CREATE TABLE sqlite_stat1(tbl,idx,stat);
sqlite> CREATE TABLE iiftest(a int, b int, c int); sqlite> EXPLAIN ANALYZE iiftest;
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     21    0                    0   Start at 21
1     Null           0     1     0                    0   r[1]=NULL
2     OpenWrite      3     4     0     3              0   root=4 iDb=0; sqlite_stat1
3     Rewind         3     9     0                    0
4       Column         3     0     2                    0   r[2]= cursor 3 column 0
5       Ne             3     8     2     BINARY-8       81  if r[2]!=r[3] goto 8
6       Rowid          3     4     0                    0   r[4]=sqlite_stat1.rowid
7       Delete         3     0     0     sqlite_stat1   2
8     Next           3     4     0                    1
9     OpenWrite      0     4     0     3              0   root=4 iDb=0; sqlite_stat1
10    OpenRead       4     2     0     3              0   root=2 iDb=0; iiftest
11    String8        0     11    0     iiftest        0   r[11]='iiftest'; iiftest
12    Count          4     13    0                    0   r[13]=count()
13    IfNot          13    18    0                    0
14    Null           0     12    0                    0   r[12]=NULL
15    MakeRecord     11    3     9     BBB            0   r[9]=mkrec(r[11..13])
16    NewRowid       0     5     0                    0   r[5]=rowid
17    Insert         0     9     5                    8   intkey=r[5] data=r[9]
18    LoadAnalysis   0     0     0                    0
19    Expire         0     0     0                    0
20    Halt           0     0     0                    0
21    Transaction    0     1     9     0              1   usesStmtJournal=0
22    String8        0     3     0     iiftest        0   r[3]='iiftest'
23    Goto           0     1     0                    0
```
Turso can now generate:
```
turso> create table sqlite_stat1(tbl,idx,stat);
turso> create table iiftest(a int, b int, c int);
turso> explain analyze iiftest;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     19    0                    0   Start at 19
1     Null               0     1     0                    0   r[1]=NULL
2     OpenWrite          0     2     0                    0   root=2; iDb=0
3     Rewind             0     9     0                    0   Rewind  sqlite_stat1
4       Column           0     0     2                    0   r[2]=sqlite_stat1.tbl
5       Ne               2     3     9                    0   if r[2]!=r[3] goto 9
6       RowId            0     4     0                    0   r[4]=sqlite_stat1.rowid
7       Delete           0     0     0     sqlite_stat1   0
8     Next               0     4     0                    0
9     OpenWrite          1     2     0                    0   root=2; iDb=0
10    OpenRead           2     3     0                    0   =iiftest, root=3, iDb=0
11    String8            0     7     0     iiftest        0   r[7]='iiftest'
12    Count              2     9     0                    0
13    IfNot              9     18    0                    0   if !r[9] goto 18
14    Null               0     8     0                    0   r[8]=NULL
15    MakeRecord         7     3     6                    0   r[6]=mkrec(r[7..9])
16    NewRowid           1     5     0                    0   r[5]=rowid
17    Insert             1     6     5     sqlite_stat1   0   intkey=r[5] data=r[6]
18    Halt               0     0     0                    0
19    String8            0     3     0     iiftest        0   r[3]='iiftest'
20    Goto               0     1     0                    0
```
Note the missing support for LoadAnalysis and Expire, but there's no
optimizer work done yet to leverage any gathered statistics yet anyway.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2740
 2025-08-23 10:50:57 +03:00
Alex Miller
4619890ffc Add basic support for ANALYZE statement. 
This permits only `ANALYZE <table_name>` to work, and all other forms
fail with a parse error (as documented in the tests).

On SQLite, ANALYZE generates:

sqlite> CREATE TABLE sqlite_stat1(tbl,idx,stat);
sqlite> CREATE TABLE iiftest(a int, b int, c int);
sqlite> EXPLAIN ANALYZE iiftest;
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     21    0                    0   Start at 21
1     Null           0     1     0                    0   r[1]=NULL
2     OpenWrite      3     4     0     3              0   root=4 iDb=0; sqlite_stat1
3     Rewind         3     9     0                    0
4       Column         3     0     2                    0   r[2]= cursor 3 column 0
5       Ne             3     8     2     BINARY-8       81  if r[2]!=r[3] goto 8
6       Rowid          3     4     0                    0   r[4]=sqlite_stat1.rowid
7       Delete         3     0     0     sqlite_stat1   2
8     Next           3     4     0                    1
9     OpenWrite      0     4     0     3              0   root=4 iDb=0; sqlite_stat1
10    OpenRead       4     2     0     3              0   root=2 iDb=0; iiftest
11    String8        0     11    0     iiftest        0   r[11]='iiftest'; iiftest
12    Count          4     13    0                    0   r[13]=count()
13    IfNot          13    18    0                    0
14    Null           0     12    0                    0   r[12]=NULL
15    MakeRecord     11    3     9     BBB            0   r[9]=mkrec(r[11..13])
16    NewRowid       0     5     0                    0   r[5]=rowid
17    Insert         0     9     5                    8   intkey=r[5] data=r[9]
18    LoadAnalysis   0     0     0                    0
19    Expire         0     0     0                    0
20    Halt           0     0     0                    0
21    Transaction    0     1     9     0              1   usesStmtJournal=0
22    String8        0     3     0     iiftest        0   r[3]='iiftest'
23    Goto           0     1     0                    0

Turso can now generate:

turso> create table sqlite_stat1(tbl,idx,stat);
turso> create table iiftest(a int, b int, c int);
turso> explain analyze iiftest;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     19    0                    0   Start at 19
1     Null               0     1     0                    0   r[1]=NULL
2     OpenWrite          0     2     0                    0   root=2; iDb=0
3     Rewind             0     9     0                    0   Rewind  sqlite_stat1
4       Column           0     0     2                    0   r[2]=sqlite_stat1.tbl
5       Ne               2     3     9                    0   if r[2]!=r[3] goto 9
6       RowId            0     4     0                    0   r[4]=sqlite_stat1.rowid
7       Delete           0     0     0     sqlite_stat1   0
8     Next               0     4     0                    0
9     OpenWrite          1     2     0                    0   root=2; iDb=0
10    OpenRead           2     3     0                    0   =iiftest, root=3, iDb=0
11    String8            0     7     0     iiftest        0   r[7]='iiftest'
12    Count              2     9     0                    0
13    IfNot              9     18    0                    0   if !r[9] goto 18
14    Null               0     8     0                    0   r[8]=NULL
15    MakeRecord         7     3     6                    0   r[6]=mkrec(r[7..9])
16    NewRowid           1     5     0                    0   r[5]=rowid
17    Insert             1     6     5     sqlite_stat1   0   intkey=r[5] data=r[6]
18    Halt               0     0     0                    0
19    String8            0     3     0     iiftest        0   r[3]='iiftest'
20    Goto               0     1     0                    0

Note the missing support for LoadAnalysis and Expire, but there's no
optimizer work done yet to leverage any gathered statistics yet anyway.
 2025-08-22 23:18:53 -07:00
Alex Miller
327936fb51 Highlight the EXPLAIN keyword 2025-08-22 22:49:58 -07:00
Alex Miller
f1250a6c86 Add syntax highlighting for ANALYZE 2025-08-22 22:47:46 -07:00
Pekka Enberg
78295e3b4c Merge 'wal-api: allow to mix frames insert with SQL execution' from Nikita Sivukhin 
This PR make it possible to do 2 pretty crazy things with turso-db:
1. Now we can mix WAL frames inserts with SQL execution within same
transaction. This will allow sync engine to execute rebase of local
changes within atomically over main database file (the operation first
require us to push new frames to physically revert local changes and
then we need to replay local logical changes on top of the modified DB
state)
2. Under `conn_raw_api` Cargo feature turso-db now expose method which
allow caller to specify WAL file path. This dangerous capability exposed
for sync-engine which maintain 2 databases: main one and "revert"-DB
which shares same DB file but has it's own separate WAL. As sync-engine
has full control over checkpoint - it can guarantee that DB file will be
consistent with both main and "revert" DB WALs.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2716
 2025-08-22 15:41:43 +03:00