Commit Graph

7879 Commits

Author SHA1 Message Date
Pekka Enberg
19456147ec testing: Add unreliable libc 2025-08-20 13:43:47 +03:00
Pekka Enberg
9233f48e08 core/io: Switch Unix I/O operations to use libc
We need it for LD_PRELOAD fault injection to work.
2025-08-20 13:43:47 +03:00
Pekka Enberg
29a2fa1dd2 Fix Antithesis Dockerfile 2025-08-20 13:43:31 +03:00
Pekka Enberg
c3ed6f570c Merge 'SQLite C API improvements: add column type and column decltype' from Danawan Bimantoro
Closes #2676
2025-08-20 11:24:43 +03:00
Pekka Enberg
9998834d3d Merge 'Fix column nullability constraint' from
Closes #2553 .

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2681
2025-08-20 11:24:21 +03:00
Jussi Saurio
19e4aa6aeb Merge 'sim: use 'git rev-parse --show-toplevel' for getting base dir' from Jussi Saurio
The current `get_limbo_project_dir()` does not work from git worktrees

Closes #2684
2025-08-20 11:16:24 +03:00
Jussi Saurio
e5f04ae100 Merge 'refactor/vdbe: move insert-related seeking to VDBE from BTreeCursor' from Jussi Saurio
This gets rid of `InsertState` in `BTreeCursor` plus the `moved_before`
parameter to `BTreeCursor::insert` -- instead, seek logic is now in the
existing state machines for `op_insert` and `op_idx_insert`

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2639
2025-08-20 11:15:09 +03:00
Pekka Enberg
c2208a542a Merge 'Initial pass to support per page encryption' from Avinash Sajjanshetty
This patch adds support for per page encryption. The code is of alpha
quality, was to test my hypothesis. All the encryption code is gated
behind a `encryption` flag. To play with it, you can do:
```sh
cargo run --features encryption -- database.db

turso> PRAGMA key='turso_test_encryption_key_123456';

turso> CREATE TABLE t(v);
```
Right now, most stuff is hard coded. We use AES GCM 256. This
information is not stored anywhere, but in future versions we will start
saving this info in the file. When writing to disk, we will generate a
cryptographically secure random salt, use that to encrypt the page. Then
we will store the authentication tag and the salt in the page itself. To
accommodate this encryption hardcodes reserved space of 28 bytes.
Once the key is set in the connection, we propagate that information to
pager and the WAL, to encrypt / decrypt when reading from disk.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2567
2025-08-20 11:11:24 +03:00
Pekka Enberg
8fcb97b088 Fix MCP server mode section formatting 2025-08-20 10:58:54 +03:00
Pekka Enberg
494ce41d80 Turso 0.1.4 2025-08-20 10:35:35 +03:00
Avinash Sajjanshetty
40a209c000 simplify feature flag usage for encryption 2025-08-20 12:49:38 +05:30
Jussi Saurio
e6adb8992b sim: use 'git rev-parse --show-toplevel' for getting base dir 2025-08-20 09:58:21 +03:00
Pekka Enberg
dc47cd83b1 Update CHANGELOG 2025-08-20 09:33:26 +03:00
Pekka Enberg
d456db31db Turso 0.1.4-pre.11 2025-08-20 09:32:10 +03:00
Pekka Enberg
0384a3a33b Merge 'Fix page locked panic' from Pedro Muniz
Clear locked pages when read completions callback fail. Also, we need to
abort I/O Completions in `stmt.run_once()` so that we do not raise
errors in `rollback` when clearing the page cache.
Fixes #2658
Fixes #2675
Fixes #2680
Fixes #2682

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2683
2025-08-20 09:31:26 +03:00
Avinash Sajjanshetty
eb45a156fc Add a simple test for encryption 2025-08-20 11:47:25 +05:30
Avinash Sajjanshetty
fc3b76de1b fix Database storage for WASM bindings 2025-08-20 11:47:25 +05:30
Avinash Sajjanshetty
bd9b4bbfd2 encrypt/decrypt when writing/reading from DB 2025-08-20 11:47:23 +05:30
Avinash Sajjanshetty
657daeded3 encrypt/decrypt when writing/reading from WAL 2025-08-20 11:44:08 +05:30
Avinash Sajjanshetty
201262b3dd Update DatabaseStorage to pass encryption context 2025-08-20 11:41:08 +05:30
Avinash Sajjanshetty
94d38be1a2 Set reserved_space to 28 for encrypted databases
We will use this space to store nonce and tag
2025-08-20 11:39:09 +05:30
Avinash Sajjanshetty
a6e9237c94 Set encryption key in pager and WAL 2025-08-20 11:39:09 +05:30
Avinash Sajjanshetty
93774ffc3b Add PRAGMA key to set the encryption key
If set, set the key for the connection
2025-08-20 11:39:07 +05:30
Avinash Sajjanshetty
100a0d8e97 Add encryption module
Let's add an encryption module, hard coded to use AES 256 GCM.
Other required parameters are also hard coded and will be made
configurable in the future PRs.

The module is behind a `encryption` feature flag.
2025-08-20 11:38:11 +05:30
danawan
45376e0b35 delete printf in C test 2025-08-20 12:01:55 +07:00
danawan
72cdd32ba1 fix null testing 2025-08-20 11:59:27 +07:00
pedrocarlo
d61d6c0872 when run_once fails we abort the current IOCompletions 2025-08-20 01:36:08 -03:00
pedrocarlo
f27d4d14f2 remove polling code in UnixIO so we can implement it correctly later and so we do not fool ourselves that we have any async code there that actually runs 2025-08-20 01:36:08 -03:00
danawan
804bb868c7 add column decltype 2025-08-20 11:28:14 +07:00
pedrocarlo
7e98a464a7 check if completion finished instead of completed for step 2025-08-20 00:38:16 -03:00
rajajisai
ff2d62aa9d update test 2025-08-19 23:11:10 -04:00
danawan
f1c2277543 change column type using value type 2025-08-20 09:34:30 +07:00
rajajisai
73500eb00c Include tests 2025-08-19 22:33:59 -04:00
rajajisai
89cd3fe196 notnull is now set based on the nullable field instead of being hardcoded. 2025-08-19 21:49:04 -04:00
danawan
85da74979e change get column type from core 2025-08-20 07:55:41 +07:00
danawan
41c0510e80 Merge branch 'main' of github.com:danawanb/turso 2025-08-20 07:38:17 +07:00
pedrocarlo
46c756b130 clear locked on pages when completion errors 2025-08-19 17:29:57 -03:00
Jussi Saurio
1614b0e0fa Merge 'Remove assertions from Completion::complete() and Completion::error()' from Jussi Saurio
The completion callback can be invoked only once via `OnceLock`, let's
not crash if we e.g. call `Completion::abort()` on an already finished
completion.
Closes #2673

Closes #2674
2025-08-19 22:28:19 +03:00
Jussi Saurio
b5439dd068 Remove assertions from Completion::complete() and Completion::error()
The completion callback can be invoked only once via `OnceLock`, let's not
crash if we e.g. call `Completion::abort()` on an already finished completion.

Closes #2673
2025-08-19 22:02:02 +03:00
Pekka Enberg
8d7ab52471 Turso 0.1.4-pre.10 2025-08-19 19:32:47 +03:00
Pekka Enberg
da80e6f760 Add links to JavaScript packages 2025-08-19 19:31:32 +03:00
Jussi Saurio
a82930d641 Merge 'Completion Error' from Pedro Muniz
Completions can now carry errors inside of them. This allows us to wait
for a completion to complete or to error. When it errors we can properly
tell the caller of `wait_for_completion` that we errored. This will also
allow us to abort completions.
Currently, this just creates the scaffold for us to store the error in
the completion. But to correctly achieve this, it will require some
refactor of our IO implementations to store the `run_once` error for a
particular completion inside of it instead of short circuiting. This
would also allow us to check for an error in `program.step` and properly
rollback.
Also, creates default impls for some common IO methods, this is
important specially for `wait_for_completion` as we want to check the
error in the `Completion` before returning `Ok`.
Maybe we could also accept a Result type in the completion callback so
that we can execute some sort of compensating action on error, like
unlocking a page so it can be evicted by the page cache later.
**EDIT:** actually implemented this in this PR. We store a `Result`
object inside `CompletionInner` behind a `OnceLock` for thread-safety.
We also pass a result object to Completion callbacks to execute
compensating actions.

Reviewed-by: Avinash Sajjanshetty (@avinassh)

Closes #2589
2025-08-19 19:07:57 +03:00
Jussi Saurio
c2855cb0db refactor/idx_insert: move seeking to VDBE instead of BTreeCursor
Also removes `InsertState` and `moved_before` since neither are
needed anymore.
2025-08-19 19:04:42 +03:00
Jussi Saurio
d191c7d98b refactor/insert: move seeking to VDBE instead of BTreeCursor 2025-08-19 19:04:20 +03:00
Pekka Enberg
6be1f67714 Merge 'JavaScript improvements' from Pekka Enberg
This pull request brings the JavaScript APIs to almost feature parity
with libSQL. Some corner cases like error messages still need some more
work.

Closes #2669
2025-08-19 18:27:51 +03:00
Pekka Enberg
945089d792 Merge 'bindings/rust: Add method' from Pekka Enberg
Fixes #2670

Closes #2671
2025-08-19 18:26:18 +03:00
Pekka Enberg
2963ea7239 bindings/rust: Add Connection::last_insert_rowid() method
Fixes #2670
2025-08-19 17:49:24 +03:00
pedrocarlo
b1399f6e8c fix merge conflicts 2025-08-19 10:48:21 -03:00
pedrocarlo
66171527b4 thread safely store the result of completion 2025-08-19 10:48:21 -03:00
pedrocarlo
de1811dea7 abort completions on error 2025-08-19 10:48:21 -03:00