Commit Graph

1265 Commits

Author SHA1 Message Date
Pekka Enberg
b9bb859271 Merge 'Switch to new parser in core' from Levy A.
Integrate #2381 to core. Resolves #2337.

Reviewed-by: Lâm Hoàng Phúc (@TcMits)

Closes #2650
2025-08-22 10:06:37 +03:00
Pekka Enberg
156352fabc Merge ' should not return a Completion when there is a page cache hit' from Pedro Muniz
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2723
2025-08-22 07:33:28 +03:00
Levy A.
4ba1304fb9 complete parser integration 2025-08-21 15:23:59 -03:00
pedrocarlo
6b0ed08465 read_page should return No Completion when have a page cache hit 2025-08-21 14:39:24 -03:00
Nikita Sivukhin
89e180c2a8 add few tests 2025-08-21 18:27:54 +04:00
Nikita Sivukhin
94f2e5a4b9 add truncate method to the page cache 2025-08-21 18:27:54 +04:00
Jussi Saurio
05f4cec34d Merge 'Wal api checkpoint seq' from Nikita Sivukhin
This PR adds information about checkpoint sequence number to the WAL raw
API. Will be used in the sync engine.
Depends on the #2699

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2707
2025-08-21 17:05:45 +03:00
Nikita Sivukhin
d7e47c1268 fix bug - continue checkpoint as usual even if frames range is degenerate 2025-08-21 17:37:19 +04:00
Jussi Saurio
4ceadcca78 Merge 'Remove hardcoded flag usage in DBHeader for encryption' from Avinash Sajjanshetty
Previously, we just hardcoded the reserved space with encryption flag.
This patch removes that and sets the reserved space if a key was
specified during a creation of db

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2706
2025-08-21 15:46:41 +03:00
Nikita Sivukhin
69c39d5d8c replace wal_frames_count with wal_state method which return both frames count and checkpoint sequence 2025-08-21 15:13:23 +04:00
Nikita Sivukhin
25cb28da67 add method to get checkpoint_seq from WAL 2025-08-21 15:13:23 +04:00
Nikita Sivukhin
38eb5232c8 do not check page size if it's not initialized yet 2025-08-21 15:12:22 +04:00
Nikita Sivukhin
10a164e399 extend checkpoint result with information about last checkpointed frame 2025-08-21 15:12:15 +04:00
Avinash Sajjanshetty
1f93e77828 Remove hardcoded flag usage in DBHeader for encryption
Previously, we just hardcoded the reserved space with encryption flag.
This patch removes that and sets the reserved space if a key was
specified during a creation of db
2025-08-21 16:21:35 +05:30
Nikita Sivukhin
05931f70ce add optional upper_bound_inclusive parameter to some checkpoint modes
- will be used in sync-engine protocol
2025-08-21 14:12:11 +04:00
PThorpe92
4a2da6c262 Remove assertion for checkpoint seq in favor of selectively using cached pages 2025-08-20 18:26:55 -04:00
PThorpe92
7082086061 Remove ENV var and enable cache by default, track which pages were cached 2025-08-20 17:42:17 -04:00
PThorpe92
345b80d14c Change env var to ENABLE instead of DISABLE so its disabled by default 2025-08-20 17:36:00 -04:00
PThorpe92
51e4cd0f1d Add debug assertion for cached pages used during checkpoint 2025-08-20 17:35:59 -04:00
PThorpe92
e28a38abc5 Fix wal tag safety issues, and add debug assertion that we are reading the proper frames 2025-08-20 17:28:48 -04:00
PThorpe92
4100737358 remove page entries without frames in frame cache in WAL rollback method 2025-08-20 17:28:19 -04:00
PThorpe92
d2c3ba14c8 Remove inefficient vec in WAL for tracking pages present in frame cache 2025-08-20 17:28:18 -04:00
PThorpe92
d6d72d2966 Update Page to carry epoch of frame + checkpont seq to ensure proper cached page for chkpt 2025-08-20 17:28:17 -04:00
PThorpe92
00f2a0f216 Performance improvements to checkpointing. prevent serializing I/O 2025-08-20 17:26:54 -04:00
PThorpe92
fe7a5e98b8 Track frame_ids on PageInner and use the page cache for reading pages to checkpoint 2025-08-20 17:24:10 -04:00
Pekka Enberg
72a5de3551 Merge 'core/mvcc: support for MVCC' from Pere Diaz Bou
This PR tries to add simple support for delete, with limited testing for
now.
Moreover, there was an error with `forward`, which wasn't obvious
without delete, which didn't skip deleted rows.

Reviewed-by: Avinash Sajjanshetty (@avinassh)

Closes #2672
2025-08-20 16:51:31 +03:00
Jussi Saurio
e5f04ae100 Merge 'refactor/vdbe: move insert-related seeking to VDBE from BTreeCursor' from Jussi Saurio
This gets rid of `InsertState` in `BTreeCursor` plus the `moved_before`
parameter to `BTreeCursor::insert` -- instead, seek logic is now in the
existing state machines for `op_insert` and `op_idx_insert`

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2639
2025-08-20 11:15:09 +03:00
Pekka Enberg
c2208a542a Merge 'Initial pass to support per page encryption' from Avinash Sajjanshetty
This patch adds support for per page encryption. The code is of alpha
quality, was to test my hypothesis. All the encryption code is gated
behind a `encryption` flag. To play with it, you can do:
```sh
cargo run --features encryption -- database.db

turso> PRAGMA key='turso_test_encryption_key_123456';

turso> CREATE TABLE t(v);
```
Right now, most stuff is hard coded. We use AES GCM 256. This
information is not stored anywhere, but in future versions we will start
saving this info in the file. When writing to disk, we will generate a
cryptographically secure random salt, use that to encrypt the page. Then
we will store the authentication tag and the salt in the page itself. To
accommodate this encryption hardcodes reserved space of 28 bytes.
Once the key is set in the connection, we propagate that information to
pager and the WAL, to encrypt / decrypt when reading from disk.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2567
2025-08-20 11:11:24 +03:00
Avinash Sajjanshetty
40a209c000 simplify feature flag usage for encryption 2025-08-20 12:49:38 +05:30
Avinash Sajjanshetty
bd9b4bbfd2 encrypt/decrypt when writing/reading from DB 2025-08-20 11:47:23 +05:30
Avinash Sajjanshetty
657daeded3 encrypt/decrypt when writing/reading from WAL 2025-08-20 11:44:08 +05:30
Avinash Sajjanshetty
201262b3dd Update DatabaseStorage to pass encryption context 2025-08-20 11:41:08 +05:30
Avinash Sajjanshetty
94d38be1a2 Set reserved_space to 28 for encrypted databases
We will use this space to store nonce and tag
2025-08-20 11:39:09 +05:30
Avinash Sajjanshetty
a6e9237c94 Set encryption key in pager and WAL 2025-08-20 11:39:09 +05:30
Avinash Sajjanshetty
100a0d8e97 Add encryption module
Let's add an encryption module, hard coded to use AES 256 GCM.
Other required parameters are also hard coded and will be made
configurable in the future PRs.

The module is behind a `encryption` feature flag.
2025-08-20 11:38:11 +05:30
pedrocarlo
46c756b130 clear locked on pages when completion errors 2025-08-19 17:29:57 -03:00
Pere Diaz Bou
4314bc13e6 core/mvcc: delete support 2025-08-19 19:48:36 +02:00
Jussi Saurio
c2855cb0db refactor/idx_insert: move seeking to VDBE instead of BTreeCursor
Also removes `InsertState` and `moved_before` since neither are
needed anymore.
2025-08-19 19:04:42 +03:00
pedrocarlo
de1811dea7 abort completions on error 2025-08-19 10:48:21 -03:00
pedrocarlo
ab3b68e360 change completion callbacks to take a Result param + create separate functions to declare a completion errored 2025-08-19 10:48:21 -03:00
Jussi Saurio
ac37e89fe1 remove unused PAGE_ERROR flag 2025-08-19 15:44:16 +03:00
Jussi Saurio
33eb730ef8 finish_read_page() never fails, so it does not need to return Result 2025-08-19 15:44:16 +03:00
Jussi Saurio
687e593361 Add pgno field to CacheError::Locked for debugging 2025-08-19 15:44:16 +03:00
Jussi Saurio
7f1eac9560 Do not start or end transaction in nested statement 2025-08-19 13:03:14 +03:00
Jussi Saurio
9deedd917f Merge 'FaultyQuery enabled by default' from Pedro Muniz
FaultyQuery was disabled. We were not simulating faults.

Reviewed-by: Avinash Sajjanshetty (@avinassh)
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2594
2025-08-18 22:53:37 +03:00
pedrocarlo
aa7f8fbfc4 Tripwire for Wal Syncing + set syncing to true before calling sync 2025-08-18 15:34:20 -03:00
pedrocarlo
c5dc26f542 reset more pager state machines in rollback 2025-08-18 12:01:10 -03:00
Pere Diaz Bou
94cd504d4c core/mvcc: support schema_did change on commit_txn
This not only changes schema_did_change on commit_txn for mvcc, but also
extracts the connection transaction state from non mvcc transactions to
mvcc too.
2025-08-18 15:52:10 +02:00
Jussi Saurio
9abc63d853 Add a bit of abstraction for creating EQ cursorcontexts 2025-08-18 13:13:02 +03:00
Jussi Saurio
3eb89982ba Remove obsolete FIXME 2025-08-18 12:08:40 +03:00