Commit Graph

6391 Commits

Author SHA1 Message Date
Pere Diaz Bou
fae0a67ff7 Merge ' core/lib: init_pager lock shared wal until filled ' from Pere Diaz Bou
maybe_shared_wal's lock is held for a limited time increasing the chance
of initializing the shared wal twice.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2201
2025-07-21 13:06:41 +02:00
Pere Diaz Bou
6dd8f6561d clippy 2025-07-21 12:53:56 +02:00
Jussi Saurio
d308f4d206 Merge 'fix/sim: actually enable indexes by default' from Jussi Saurio
Follow up to #2199 that _actually_ enables indexes by default in sim

Closes #2202
2025-07-21 13:44:51 +03:00
Pere Diaz Bou
4c59695351 test_multi_thread: ignore tests for now 2025-07-21 12:43:31 +02:00
Jussi Saurio
e604a80e93 fix/sim: actually enable indexes by default 2025-07-21 13:35:11 +03:00
Pekka Enberg
87f0229e1f Merge 'test/stress&sim: enable indexes by default' from Jussi Saurio
Life is too boring without any simulator-created issues

Closes #2199
2025-07-21 12:39:57 +03:00
Pere Diaz Bou
c55cb74dc8 simple write multi threaded test 2025-07-21 11:06:31 +02:00
Pere Diaz Bou
e3fdb6bab9 core/lib: init_pager lock shared wal until filled
maybe_shared_wal's lock is held for a limited time increasing the chance
of initializing the shared wal twice.
2025-07-21 11:00:34 +02:00
Jussi Saurio
dce355d9b7 test/stress&sim: enable indexes by default 2025-07-21 11:49:30 +03:00
Jussi Saurio
d6bd9fc26e Merge 'fix/btree/balance: interior cell insertion can leave page overfull' from Jussi Saurio
- When an interior index cell is replaced, it can cause the page where
the
replacement happens to overflow OR underflow. On `main` we did not check
this case, because
the interior cell replacement always moves the cursor to a leaf, and if
the leaf
doesn't underflow, then no further balancing happens.
- The solution is to ALWAYS check whether the interior page where the
replacement
happens is underflowing OR overflowing, and balance that page regardless
of whether
the leaf page where the replacement was taken underflows or not.
So summary:
- InteriorCellReplacement: cell deleted from Interior page I,
replacement cell taken from Leaf L
  and inserted back to Interior page I.
- If Leaf L underflows:
  * balance it first
  * then balance I if it overflows OR underflows
- If Leaf L does NOT underflow:
  * balance I anyway if it overflows OR underflows
Closes https://github.com/tursodatabase/turso/issues/1701
Closes https://github.com/tursodatabase/turso/issues/2167

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #2168
2025-07-21 11:03:26 +03:00
Jussi Saurio
079a9fe0a5 Merge 'Fix error: Stable 1.88.0 is not available in Nix flake' from Levy A.
Fixed by just updating with `nix flake update`.
```
error:
       … while calling the 'derivationStrict' builtin
         at <nix/derivation-internal.nix>:37:12:
           36|
           37|   strict = derivationStrict drvAttrs;
             |            ^
           38|

       … while evaluating derivation 'nix-shell'
         whose name attribute is located at /nix/store/wshnc0kqk1qz7iffb1yqri8a5cy6v7w5-source/pkgs/stdenv/generic/make-derivation.nix:375:7

       … while evaluating attribute '__impureHostDeps' of derivation 'nix-shell'
         at /nix/store/wshnc0kqk1qz7iffb1yqri8a5cy6v7w5-source/pkgs/stdenv/generic/make-derivation.nix:490:7:
          489|       __propagatedSandboxProfile = unique (computedPropagatedSandboxProfile ++ [ propagatedSandboxProfile ]);
          490|       __impureHostDeps = computedImpureHostDeps ++ computedPropagatedImpureHostDeps ++ __propagatedImpureHostDeps ++ __impureHostDeps ++ stdenv.__extraImpureHostDeps ++ [
             |       ^
          491|         "/dev/zero"

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: Stable 1.88.0 is not available
```

Closes #2150
2025-07-21 09:44:21 +03:00
Jussi Saurio
2967fafe73 Merge 'Usable space unwrap' from Pedro Muniz
Using `unwrap_or_default` can make `page_size` become 0 in this case,
which can lead to subtracting with overflow in `payload_threshold_max`
in case we have some sort of error. Better to unwrap the error here, as
in release mode we may not have overflow checks enabled to catch this.

Closes #2145
2025-07-21 00:23:06 +03:00
Jussi Saurio
cff4d7e16d Merge 'BTreeTable::to_sql: wrap special column names in brackets' from Nils Koch
Closes #2055

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2126
2025-07-21 00:22:49 +03:00
Jussi Saurio
026cb7390c Merge 'fix/test: fix and unignore incorrectly implemented test' from Jussi Saurio
Turns out #2192 was just me implementing a test wrong -- `BEGIN` is ofc
an alias for `BEGIN DEFERRED` so just executing `BEGIN` won't start a tx
yet. In short: `tursodb` does what you'd expect under snapshot
isolation, at least in this specific instance.
In any case, let's not waste the test -- I've now unignored it and fixed
it.

Closes #2196
2025-07-21 00:06:18 +03:00
Jussi Saurio
81d8dc7e3d Merge 'Fix column order for multi-row insertion' from Nikita Sivukhin
Column order handled incorrectly for multi-rows insert case
```sql
turso> CREATE TABLE t(x, y, z);
turso> INSERT INTO t(z, x) VALUES (1, 2), (3, 4);
turso> SELECT * FROM t;
┌───┬───┬───┐
│ x │ y │ z │
├───┼───┼───┤
│   │ 2 │   │
├───┼───┼───┤
│   │ 4 │   │
└───┴───┴───┘
```

Closes #2109
2025-07-21 00:02:43 +03:00
Jussi Saurio
9936748132 Merge 'Avoid redundant decoding of record headers when reading sorted chunk files' from Iaroslav Zeigerman
Currently, each record header is decoded at least twice: once to
determine the record size within the read buffer (in order to construct
the `ImmutableRecord` instance), and again later when decoding the
record for comparison. This redundant decoding can have a noticeable
negative impact on performance when records are wide (eg. contain
multiple columns).
This update modifies the (de)serialization format for sorted chunk files
by prepending a record size varint to each record payload. As a result,
only a single varint needs to be decoded to determine the record size,
eliminating the need to decode the full record header during reads.

Closes #2176
2025-07-20 23:54:54 +03:00
Jussi Saurio
8a6f3f589c Merge 'compat: change page_size pragma and RowData opcode to yes' from meteorgan
page_size pragma: #2053
RowData opcode: #1756

Closes #2187
2025-07-20 23:53:11 +03:00
Jussi Saurio
34f91fc727 Merge 'doc: fix intra-repo links for contribution guide and license, and expand binding links' from bit-aloo
Small README cleanup:
1. Updated the contribution guide and license link to point to the local
CONTRIBUTING.md and LICENSE.md file instead of GitHub’s default page.
2. Linked each language in the bindings list to its respective
subdirectory for easier navigation.

Closes #2190
2025-07-20 23:52:45 +03:00
Jussi Saurio
a9a43f5644 Merge 'Explain the Turso challenge' from Glauber Costa
The challenge is not being mentioned where it matters most: in the
Github Readme, where developers are!
Fix it.

Closes #2193
2025-07-20 23:52:09 +03:00
Jussi Saurio
21d1781ef9 fix/test: fix and unignore incorrectly implemented test 2025-07-20 23:48:54 +03:00
Jussi Saurio
0987618d6b fix/btree/balance: interior cell insertion can leave page unbalanced
- When an interior index cell is replaced, it can cause the page where the
replacement happens to overflow. On `main` we did not check this case, because
the interior cell replacement always moves the cursor to a leaf, and if the leaf
doesn't underflow, then no further balancing happens.

- The solution is to ALWAYS check whether the interior page where the replacement
happens is underflowing OR overflowing, and balance that page regardless of whether
the leaf page where the replacement was taken underflows or not.

So summary:

- InteriorCellReplacement: cell deleted from Interior page I, replacement cell taken from Leaf L
  and inserted back to Interior page I.
- If Leaf L underflows:
  * balance it first
  * then balance I if it overflows OR underflows
- If Leaf L does NOT underflow:
  * balance I anyway

Closes #1701
Closes #2167
2025-07-20 23:38:47 +03:00
Nils Koch
ed8600db48 add test for dropping special column names 2025-07-20 21:20:59 +01:00
Nils Koch
05a9acf8c5 wrap special column names with [] in BTreeTable to_sql 2025-07-20 21:20:59 +01:00
Pere Diaz Bou
90f71ccd4e Merge 'fix opcodes missing a database register' from Glauber Costa
Two of the opcodes we implement (OpenRead and Transaction) should have
an opcode specifying the database to use, but they don't.
Add it, and for now always use 0 (the main database).

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #2191
2025-07-20 21:01:35 +02:00
Pere Diaz Bou
a7662280cd Merge 'fix/pager/cacheflush: cacheflush shouldn't commit' from Jussi Saurio
Closes #2188
Closes #2194
Doesn't fix #2192 but adds an ignored test

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #2189
2025-07-20 20:58:27 +02:00
Jussi Saurio
010fb1c12a fix/pager/cacheflush: cacheflush shouldn't commit 2025-07-20 21:18:45 +03:00
Glauber Costa
eaff8cb824 Explain the Turso challenge
The challenge is not being mentioned where it matters most: in the
Github Readme, where developers are!

Fix it.
2025-07-20 12:57:23 -05:00
Glauber Costa
65312baee6 fix opcodes missing a database register
Two of the opcodes we implement (OpenRead and Transaction) should have
an opcode specifying the database to use, but they don't.

Add it, and for now always use 0 (the main database).
2025-07-20 12:27:26 -05:00
bit-aloo
5be10bb5bc doc: fix broken contribution guide link and expand language binding links 2025-07-20 22:18:59 +05:30
meteorgan
21134446cc compat: change page_size pragma and rowdata opcode to yes 2025-07-20 23:07:04 +08:00
Nikita Sivukhin
66580f42cf add test 2025-07-20 14:22:49 +04:00
Nikita Sivukhin
e733365057 fix code generation for insert command 2025-07-20 14:22:16 +04:00
Pekka Enberg
440ff43a72 Merge 'gh workflow for dart (test, precompile, publish), only test is activated' from Andika Tanuwijaya
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1957
2025-07-20 09:50:10 +03:00
Pekka Enberg
55b5e45231 Merge 'bindings/rust: Return number of rows changed from Connection::execute()' from Rohith Suresh
Fixes #1904
This PR changes the existing behaviour of Connection.execute to not
return 0, but the number of rows that have been changed by the operation
within. The changes are:
1. Adds a getter for n_change and the execute function now returns the
n_change value
2. Integration test  to test the behaviour

Closes #1987
2025-07-20 09:49:51 +03:00
Pekka Enberg
744f636c33 Merge 'bindings/java: Make TursoDB and TursoDB factory thread-safe' from Mikaël Francoeur
This PR makes `TursoDB` and `TursoDBFactory` thread-safe. I also used
the opportunity to do some minor improvements.

Reviewed-by: Kim Seon Woo (@seonWKim)

Closes #2070
2025-07-20 09:48:53 +03:00
Pekka Enberg
4be6772e8e Merge 'implement Debug for Database' from Glauber Costa
Very useful in printing data structures containing databases, like maps
Example output:
Connecting to Database { path: "sq.db", open_flags: OpenFlags(1),
db_state: "initialized", mv_store: "none", init_lock: "unlocked",
wal_state: "present", page_cache: "( capacity 100000, used: 0 )" }

Reviewed-by: Pedro Muniz (@pedrocarlo)
Reviewed-by: bit-aloo (@Shourya742)

Closes #2175
2025-07-20 09:46:09 +03:00
Pekka Enberg
068a7bbe43 Merge 'implement pragma application_id' from Glauber Costa
Just for completeness, because it is easy.

Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #2180
2025-07-20 09:22:20 +03:00
Pekka Enberg
01c7d66447 Merge 'use wasm32-wasip1 target instead of wasm32-wasi' from Nils Koch
In Rust 1.84.0, the support for the named target `wasm32-wasi` was
removed in favor of `wasm32-wasip1` ([Release
Notes](https://releases.rs/docs/1.84.0/#compatibility-notes)).
I missed that in https://github.com/tursodatabase/turso/pull/1807.
I got the following error when I tried to run `make`:
```
> make
Checking Rust version...
Rust version 1.88.0 is acceptable.
Checking wasm32-wasi target...
Installing wasm32-wasi target...
error: toolchain '1.88.0-aarch64-apple-darwin' does not support target 'wasm32-wasi'; did you mean 'wasm32-wasip1'?
note: you can see a list of supported targets with `rustc --print=target-list`
note: if you are adding support for a new target to rustc itself, see https://rustc-dev-guide.rust-lang.org/building/new-target.html
make: *** [check-wasm-target] Error 1
```

Reviewed-by: bit-aloo (@Shourya742)

Closes #2177
2025-07-20 09:21:28 +03:00
Pekka Enberg
b6eaf3262d Merge 'implement write side of pragma schema_version' from Glauber Costa
It is insane that SQLite even allows this.
They actually don't if "defensive mode" is enabled:
"It is always safe to read the schema_version, but changing the
schema_version can cause problems. For this reason, attempts to change
the value of schema_version are a silent no-op when defensive mode is
enabled for a database connection.
Warning: Misuse of this pragma can result in database corruption."
We also update the compat table, which was not updated to reflect the
read version of this pragma being implemented.

Closes #2181
2025-07-20 09:20:23 +03:00
RS2007
cd31119972 Fix: cargo clippy warning 2025-07-20 07:28:14 +05:30
RS2007
adf70dc855 Return number of rows changed from Connection.execute 2025-07-20 07:27:38 +05:30
Glauber Costa
6506b3147d implement pragma application_id
Just for completeness, because it is easy.
2025-07-19 20:44:06 -05:00
Glauber Costa
024d79fc0d implement write side of pragma schema_version
It is insane that SQLite even allows this.
They actually don't if "defensive mode" is enabled:

"It is always safe to read the schema_version, but changing the
schema_version can cause problems. For this reason, attempts to change
the value of schema_version are a silent no-op when defensive mode is
enabled for a database connection.

Warning: Misuse of this pragma can result in database corruption."

We also update the compat table, which was not updated to reflect
the read version of this pragma being implemented.
2025-07-19 20:39:30 -05:00
Iaroslav Zeigerman
10a848fbc5 address nit 2025-07-19 18:40:43 +02:00
Glauber Costa
4749ce95c1 implement Debug for Database
Very useful in printing data structures containing databases, like maps

Example output:

Connecting to Database { path: "sq.db", open_flags: OpenFlags(1), db_state: "initialized", mv_store: "none", init_lock: "unlocked", wal_state: "present", page_cache: "( capacity 100000, used: 0 )" }
2025-07-19 09:29:46 -05:00
Nils Koch
bab8001a9c use wasm32-wasip1 target instead of wasm32-wasi 2025-07-19 13:16:46 +01:00
Pekka Enberg
d1fdc7dbc8 Merge 'Simplify blocking operations – add io.block(fn) for IO trait implementors' from Levy A.
This PR make it simpler to block async operations instead of creating
sync equivalent functions. eg.: `get_page_size` and
`get_page_size_async`. All functions should be async by default, and
making them block should more explicit.
Also makes it easier to grep for top-level uses of blocking code without
needing to "go to definition".

Closes #2173
2025-07-19 12:20:07 +03:00
Pekka Enberg
16958f1a44 Turso 0.1.3-pre.3 2025-07-19 08:44:22 +03:00
Levy A.
0ea7849dca feat: IOExt utility trait 2025-07-19 01:40:42 -03:00
Iaroslav Zeigerman
5d47502e3a Avoid redundant decoding of record headers when reading sorted chunk files 2025-07-19 06:08:27 +02:00