Commit Graph

4415 Commits

Author SHA1 Message Date
Jussi Saurio
4ea8cd0007 refactor/btree: rewrite the free_cell_range() function
i had a rough time reading this function earlier and trying to understand it,
so rewrote it in a way that, to me, is much more readable.
2025-08-25 09:41:44 +03:00
Jussi Saurio
f2598a2dea Merge 'Remove Result from signature' from Mikaël Francoeur
This PR removes `Result<()>` from `Jsonb::write_to_string()`, since it
wasn't required. The method now returns `()`.

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #2717
2025-08-25 09:17:37 +03:00
Pekka Enberg
2e11e1bb8c Merge 'Switch to F_FULLSYNC on Darwin' from
Closes #2743 .

Closes #2766
2025-08-25 08:09:29 +03:00
rajajisai
9068a29380 Use unsafe block 2025-08-24 18:56:05 -04:00
rajajisai
84d20ba60f Use F_FULLSYNC in darwin based operating systems 2025-08-24 18:45:46 -04:00
Pekka Enberg
de16c770c3 Merge 'Remove duplicated attribute in ' from bit-aloo
Removed the redundant `#![allow(clippy::arc_with_non_send_sync)]` from
wal.rs, since it’s already defined in `mod.rs`.

Closes #2764
2025-08-24 20:54:26 +03:00
bit-aloo
37cebb0669 fix(clippy): remove duplicate arc_with_non_send_sync attribute in wal.rs 2025-08-24 22:59:47 +05:30
Pekka Enberg
c428ff06b2 sqlite3: Implement sqlite3_bind_parameter_index() 2025-08-24 20:10:31 +03:00
Pekka Enberg
9d2f26bb04 sqlite3: Implement sqlite3_clear_bindings() 2025-08-24 19:33:18 +03:00
Avinash Sajjanshetty
011f878158 make clippy bro happy 2025-08-24 16:21:06 +05:30
Avinash Sajjanshetty
a4b9c33b81 Use the new API to init cipher 2025-08-24 16:15:13 +05:30
Avinash Sajjanshetty
53f9c0dc7a Add support for lord AEGIS, the fastest and the greatest 2025-08-24 16:15:11 +05:30
Pekka Enberg
22c9cb6618 s/PerConnEncryptionContext/EncryptionContext/ 2025-08-24 08:17:20 +03:00
Pekka Enberg
1b89273f10 Merge 'refactor encryption module and make it configurable' from Avinash Sajjanshetty
Previously, the encryption module had hardcoded a lot of things. This
refactor makes it slightly nice and makes it configurable.
Right now cipher algorithm is assumed and hardcoded, I will make that
configurable in the upcoming PR

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2722
2025-08-24 08:16:28 +03:00
Preston Thorpe
b113c497aa Merge 'Replace a couple refcells for types that trivially impl Copy' from Preston Thorpe
Closes #2750
2025-08-23 16:33:07 -04:00
PThorpe92
9a418f1d3e Replace a couple refcells with cell in pager 2025-08-23 15:55:01 -04:00
themixednuts
80eca66be9 fix: normalize quotes in update
fixes: #2744
2025-08-23 03:17:03 -05:00
Alex Miller
4619890ffc Add basic support for ANALYZE statement.
This permits only `ANALYZE <table_name>` to work, and all other forms
fail with a parse error (as documented in the tests).

On SQLite, ANALYZE generates:

sqlite> CREATE TABLE sqlite_stat1(tbl,idx,stat);
sqlite> CREATE TABLE iiftest(a int, b int, c int);
sqlite> EXPLAIN ANALYZE iiftest;
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     21    0                    0   Start at 21
1     Null           0     1     0                    0   r[1]=NULL
2     OpenWrite      3     4     0     3              0   root=4 iDb=0; sqlite_stat1
3     Rewind         3     9     0                    0
4       Column         3     0     2                    0   r[2]= cursor 3 column 0
5       Ne             3     8     2     BINARY-8       81  if r[2]!=r[3] goto 8
6       Rowid          3     4     0                    0   r[4]=sqlite_stat1.rowid
7       Delete         3     0     0     sqlite_stat1   2
8     Next           3     4     0                    1
9     OpenWrite      0     4     0     3              0   root=4 iDb=0; sqlite_stat1
10    OpenRead       4     2     0     3              0   root=2 iDb=0; iiftest
11    String8        0     11    0     iiftest        0   r[11]='iiftest'; iiftest
12    Count          4     13    0                    0   r[13]=count()
13    IfNot          13    18    0                    0
14    Null           0     12    0                    0   r[12]=NULL
15    MakeRecord     11    3     9     BBB            0   r[9]=mkrec(r[11..13])
16    NewRowid       0     5     0                    0   r[5]=rowid
17    Insert         0     9     5                    8   intkey=r[5] data=r[9]
18    LoadAnalysis   0     0     0                    0
19    Expire         0     0     0                    0
20    Halt           0     0     0                    0
21    Transaction    0     1     9     0              1   usesStmtJournal=0
22    String8        0     3     0     iiftest        0   r[3]='iiftest'
23    Goto           0     1     0                    0

Turso can now generate:

turso> create table sqlite_stat1(tbl,idx,stat);
turso> create table iiftest(a int, b int, c int);
turso> explain analyze iiftest;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     19    0                    0   Start at 19
1     Null               0     1     0                    0   r[1]=NULL
2     OpenWrite          0     2     0                    0   root=2; iDb=0
3     Rewind             0     9     0                    0   Rewind  sqlite_stat1
4       Column           0     0     2                    0   r[2]=sqlite_stat1.tbl
5       Ne               2     3     9                    0   if r[2]!=r[3] goto 9
6       RowId            0     4     0                    0   r[4]=sqlite_stat1.rowid
7       Delete           0     0     0     sqlite_stat1   0
8     Next               0     4     0                    0
9     OpenWrite          1     2     0                    0   root=2; iDb=0
10    OpenRead           2     3     0                    0   =iiftest, root=3, iDb=0
11    String8            0     7     0     iiftest        0   r[7]='iiftest'
12    Count              2     9     0                    0
13    IfNot              9     18    0                    0   if !r[9] goto 18
14    Null               0     8     0                    0   r[8]=NULL
15    MakeRecord         7     3     6                    0   r[6]=mkrec(r[7..9])
16    NewRowid           1     5     0                    0   r[5]=rowid
17    Insert             1     6     5     sqlite_stat1   0   intkey=r[5] data=r[6]
18    Halt               0     0     0                    0
19    String8            0     3     0     iiftest        0   r[3]='iiftest'
20    Goto               0     1     0                    0

Note the missing support for LoadAnalysis and Expire, but there's no
optimizer work done yet to leverage any gathered statistics yet anyway.
2025-08-22 23:18:53 -07:00
Pekka Enberg
78295e3b4c Merge 'wal-api: allow to mix frames insert with SQL execution' from Nikita Sivukhin
This PR make it possible to do 2 pretty crazy things with turso-db:
1. Now we can mix WAL frames inserts with SQL execution within same
transaction. This will allow sync engine to execute rebase of local
changes within atomically over main database file (the operation first
require us to push new frames to physically revert local changes and
then we need to replay local logical changes on top of the modified DB
state)
2. Under `conn_raw_api` Cargo feature turso-db now expose method which
allow caller to specify WAL file path. This dangerous capability exposed
for sync-engine which maintain 2 databases: main one and "revert"-DB
which shares same DB file but has it's own separate WAL. As sync-engine
has full control over checkpoint - it can guarantee that DB file will be
consistent with both main and "revert" DB WALs.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2716
2025-08-22 15:41:43 +03:00
Jussi Saurio
14873c76fb unixio: use Mutex::lock() instead of Mutex::try_lock()
we should wait to obtain the lock, not immediately fail if we cant.
2025-08-22 10:47:50 +03:00
Pekka Enberg
b9bb859271 Merge 'Switch to new parser in core' from Levy A.
Integrate #2381 to core. Resolves #2337.

Reviewed-by: Lâm Hoàng Phúc (@TcMits)

Closes #2650
2025-08-22 10:06:37 +03:00
Pekka Enberg
156352fabc Merge ' should not return a Completion when there is a page cache hit' from Pedro Muniz
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2723
2025-08-22 07:33:28 +03:00
Jussi Saurio
4edbf3aac0 Merge 'Page cache truncate' from Nikita Sivukhin
Add `truncate` method in the page cache which remove all entries which
reference pages greater than new DB size.
This will be used in the sync engine as in its case DB size can shrink
when we "rebase" changes from remote to local.
It stands on the #2707 because touch few files from that PR

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2711
2025-08-21 23:34:39 +03:00
Levy A.
ee12ef9fb5 remove unnecessary Box<ast::Select> 2025-08-21 17:20:25 -03:00
Mikaël Francoeur
5ed2abf23f remove Result from signature 2025-08-21 14:45:00 -04:00
Levy A.
07975603d3 fix: incorrect sql statement in parser test 2025-08-21 15:24:01 -03:00
Levy A.
4ba1304fb9 complete parser integration 2025-08-21 15:23:59 -03:00
Levy A.
186e2f5d8e switch to new parser 2025-08-21 15:19:16 -03:00
pedrocarlo
6b0ed08465 read_page should return No Completion when have a page cache hit 2025-08-21 14:39:24 -03:00
Avinash Sajjanshetty
3090545167 use encryption ctx instead of encryption key 2025-08-21 22:36:32 +05:30
Avinash Sajjanshetty
cc8c763942 refactor encryption module and make it configurable 2025-08-21 22:36:31 +05:30
Pekka Enberg
ae8b1eb00d Merge 'core/io: Don't open file as non-blocking in Unix backend' from Pekka Enberg
The Unix backend is a syscall()-based, blocking implementation. The
O_NONBLOCK adds nothing.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2708
2025-08-21 19:13:39 +03:00
Nikita Sivukhin
94ed92ca4e fix compilation and clippy 2025-08-21 18:28:16 +04:00
Nikita Sivukhin
4653f78356 fix compilation 2025-08-21 18:28:16 +04:00
Nikita Sivukhin
7720e07931 allow to mix frames push with sql execution in the single WAL session
- we will use this in the sync engine
2025-08-21 18:28:16 +04:00
Nikita Sivukhin
732d998618 allow to open DB with different WAL file :mindblow: 2025-08-21 18:28:16 +04:00
Nikita Sivukhin
89e180c2a8 add few tests 2025-08-21 18:27:54 +04:00
Nikita Sivukhin
94f2e5a4b9 add truncate method to the page cache 2025-08-21 18:27:54 +04:00
Jussi Saurio
05f4cec34d Merge 'Wal api checkpoint seq' from Nikita Sivukhin
This PR adds information about checkpoint sequence number to the WAL raw
API. Will be used in the sync engine.
Depends on the #2699

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2707
2025-08-21 17:05:45 +03:00
Jussi Saurio
cc28b8833e Fix condition that checks table.cols against number of provided values 2025-08-21 16:40:10 +03:00
Jussi Saurio
b5bd31a47b Remove old unused data structures and functions 2025-08-21 16:40:10 +03:00
Jussi Saurio
ac56d5bb67 Use new datastructures and functions in translate_insert 2025-08-21 16:40:10 +03:00
Jussi Saurio
88c4eae63e Add functions for constructing and translating Insertions 2025-08-21 16:40:10 +03:00
Jussi Saurio
630441e270 Add new Insertion datastructures 2025-08-21 16:40:10 +03:00
Jussi Saurio
215485d403 Add Table::get_column_by_name method 2025-08-21 16:40:10 +03:00
Jussi Saurio
dd2e0ea596 Fix: always emit rowid when column is rowid alias
SQLite does not store the rowid alias column in the record at all
when it is a rowid alias, because the rowid is always stored anyway
in the record header.
2025-08-21 16:40:10 +03:00
Jussi Saurio
f9ad43a3a3 Merge 'Fix: all indexes need to be updated if the rowid changes' from Jussi Saurio
Found when running simulator in #2641
All indexes store the rowid as the last column, so whenever the rowid of
a given row changes the index entry must also be deleted and reinserted
with the new index.

Reviewed-by: Nikita Sivukhin (@sivukhin)

Closes #2712
2025-08-21 16:40:03 +03:00
Nikita Sivukhin
d7e47c1268 fix bug - continue checkpoint as usual even if frames range is degenerate 2025-08-21 17:37:19 +04:00
Jussi Saurio
e224bb15a8 Fix incorrect UNIQUE constraint failure behavior in UPDATE
UPDATE should skip over the UNIQUE constraint failure if the existing
row it found during the check has the same rowid as the row we are
currently updating
2025-08-21 16:30:34 +03:00
Jussi Saurio
e7fbeba120 Merge 'Fix: in UPDATE, insert rowid into index instead of NULL' from Jussi Saurio
Same deal as #2700, except this time in UPDATE. Nothing tests this on
`main` so not caught.
I will later put #2641 into mergeable condition so it will catch all of
these going forward.

Reviewed-by: Nikita Sivukhin (@sivukhin)

Closes #2710
2025-08-21 15:49:55 +03:00