Commit Graph

7039 Commits

Author SHA1 Message Date
Jussi Saurio
cd79d2dce5 test/fuzz: add ALTER TABLE column ops to tx isolation fuzz test 2025-08-05 10:05:56 +03:00
Jussi Saurio
685615dc98 test/fuzz/txn: remove assumption about hardcoded column count 2025-08-05 10:04:25 +03:00
Jussi Saurio
506bb5f67f Merge 'Direct schema mutation – add instruction' from Levy A.
Resolves #2378.
```
`ALTER TABLE _ RENAME TO _`/limbo_rename_table/
                        time:   [15.645 ms 15.741 ms 15.850 ms]
Found 12 outliers among 100 measurements (12.00%)
  8 (8.00%) high mild
  4 (4.00%) high severe
`ALTER TABLE _ RENAME TO _`/sqlite_rename_table/
                        time:   [34.728 ms 35.260 ms 35.955 ms]
Found 15 outliers among 100 measurements (15.00%)
  8 (8.00%) high mild
  7 (7.00%) high severe
  ```
<img width="1000" height="199" alt="image" src="https://github.com/user-
attachments/assets/ad943355-b57d-43d9-8a84-850461b8af41" />

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2399
2025-08-04 16:55:38 +03:00
Jussi Saurio
1813171b91 Merge 'Use pwrite for single buffer pwritev call in unix IO' from Preston Thorpe
Closes #2416
2025-08-04 16:52:14 +03:00
Jussi Saurio
5a06411ce6 Merge 'fix/core/translate: ALTER TABLE DROP COLUMN: ensure schema cookie is updated even when target table is empty' from Jussi Saurio
Closes #2431
Discovered while fuzzing #2086
## What
We update `schema_version` whenever the schema changes
## Problem
Probably unintentionally, we were calling `SetCookie` in a loop for each
row in the target table, instead of only once at the end. This means 2
things:
- For large `n`, this is a lot of unnecessary instructions
- For `n==0`, `SetCookie` doesn't get called at all -> the schema won't
get marked as having been updated -> conns can operate on a stale schema
## Fix
Lift `SetCookie` out of the loop

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2432
2025-08-04 16:51:24 +03:00
Jussi Saurio
8a1723b3c8 fix/core/translate: ALTER TABLE DROP COLUMN: ensure schema cookie is updated even when target table is empty 2025-08-04 15:05:00 +03:00
Pekka Enberg
e4accdc29d Merge 'hide dangerous methods behind conn_raw_api feature' from Nikita Sivukhin
WAL API shouldn't be exposed by default because this is relatively
dangerous API which we use internally and ordinary users shouldn't not
be interested in it.

Reviewed-by: Pekka Enberg <penberg@iki.fi>

Closes #2424
2025-08-04 14:52:40 +03:00
Pekka Enberg
1572285ee6 Merge 'preserve files in IO memory backend' from Nikita Sivukhin
Simple PR to preserve and reuse files in memory IO

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2428
2025-08-04 14:52:24 +03:00
Pekka Enberg
5b5bc441b7 Merge ' core/mvcc: fix new rowid on restart' from Pere Diaz Bou
Next rowid was being tracked globally for all tables and restarted to 0
every time database was opened

Closes #2425
2025-08-04 14:37:13 +03:00
Nikita Sivukhin
129895f0b2 preserve files in IO memory backend 2025-08-04 15:22:04 +04:00
Pere Diaz Bou
56240ddac9 core/mvcc: add restart tests 2025-08-04 12:31:17 +02:00
Pere Diaz Bou
f26e442597 core/mvcc: fix new rowid
next rowid was being tracked globally for all tables and restarted to 0
every time database was opened
2025-08-04 12:31:17 +02:00
Pere Diaz Bou
83a658d3d6 core/mvcc: add option to test with a random file and restart it 2025-08-04 12:31:17 +02:00
Nikita Sivukhin
83b1e99a61 fix compilation 2025-08-04 12:53:07 +04:00
Nikita Sivukhin
0adb40534c hind dangerous methods behind conn_raw_api feature 2025-08-04 12:40:28 +04:00
Pekka Enberg
deec70e541 Merge 'Improve SQLite3 TCL test suite' from Pekka Enberg
Add more stubs in tester.tcl so that the test suite does not bail out
early.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2405
2025-08-04 08:43:14 +03:00
Pekka Enberg
ca14799da5 Merge 'Make completions idempotent' from Preston Thorpe
Closes #2417
2025-08-04 08:42:42 +03:00
Pekka Enberg
3691b51039 Merge 'perf/btree: skip seek in move_to_rightmost() if we are already on rightmost page' from Jussi Saurio
## Background
When we get a new rowid using `op_new_rowid()`, we move to the end of
the btree to look at what the maximum rowid currently is, and then
increment it by one.
This requires a btree seek.
## Problem
If we were already on the rightmost page, this is a lot of unnecessary
work, including potentially a few page reads from disk (although to be
fair the ancestor pages are very likely to be in cache at this point.)
## Fix
Cache the rightmost page id whenever we enter it in
`move_to_rightmost()`, and invalidate it whenever we do a balancing
operation.
## Local benchmark results
```sql
Insert rows in batches/limbo_insert_1_rows
                        time:   [23.333 µs 27.718 µs 35.801 µs]
                        change: [-7.7924% +0.8805% +12.841%] (p = 0.91 > 0.05)
                        No change in performance detected.
Insert rows in batches/limbo_insert_10_rows
                        time:   [38.204 µs 38.381 µs 38.568 µs]
                        change: [-8.7188% -7.4786% -6.1955%] (p = 0.00 < 0.05)
                        Performance has improved.
Insert rows in batches/limbo_insert_100_rows
                        time:   [158.39 µs 165.06 µs 178.37 µs]
                        change: [-21.000% -18.789% -15.666%] (p = 0.00 < 0.05)
                        Performance has improved.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2409
2025-08-04 08:41:51 +03:00
PThorpe92
79629daff4 Make completions idempotent 2025-08-02 21:48:39 -04:00
Levy A.
b9a3a93ef0 fix: clippy 2025-08-02 20:06:05 -03:00
PThorpe92
b5117ac5c7 Use pwrite for single buffer in unix IO 2025-08-02 18:34:16 -04:00
Levy A.
b14a11a2fd fix: change name for schema btree + fix benchmark 2025-08-02 17:17:36 -03:00
Jussi Saurio
130e1f80ea fix/vdbe: call seek_to_last() only once in op_new_rowid 2025-08-02 14:18:58 +03:00
Jussi Saurio
63a5ef596b perf/btree: skip seek in move_to_rightmost() if we are already on rightmost page 2025-08-02 13:56:59 +03:00
Pekka Enberg
a4dcbeb392 Update PERF.md 2025-08-02 13:37:42 +03:00
Jussi Saurio
3b0c8b08fe Merge 'perf/pager: dont clear page cache on commit' from Jussi Saurio
This should be safe to do as:
1. page cache is private per connection
2. since this connection wrote the flushed pages/frames, they are up to
date from its perspective
3. multiple concurrent statements inside one connection are not
snapshot-transactional even in sqlite

Reviewed-by: Pekka Enberg <penberg@iki.fi>

Closes #2407
2025-08-02 13:35:57 +03:00
Jussi Saurio
4497d22d3f perf/pager: dont clear page cache on commit 2025-08-02 13:09:36 +03:00
Pekka Enberg
5037e2dc0d testing/sqlite3: Update all.test 2025-08-02 13:00:18 +03:00
Pekka Enberg
779b8e0149 testing/sqlite3: Import more join test cases 2025-08-02 12:59:11 +03:00
Pekka Enberg
c7497d55b6 testing/sqlite3: Disable select9.test
...it seems to run forever.
2025-08-02 12:59:11 +03:00
Pekka Enberg
a380e0775d testing/sqlite3: Improve tester.tcl
Improve tester.tcl by adding stubs so that we can run more of the test
suite without the test harness giving up.
2025-08-02 12:59:11 +03:00
Pekka Enberg
bca9994cf6 Merge 'Rename liblimbo_sqlite3 to libturso_sqlite3' from Pekka Enberg
Closes #2403
2025-08-02 12:34:19 +03:00
Pekka Enberg
12455c6531 Merge 'core: Fold HeaderRef to pager module' from Pekka Enberg
Closes #2401
2025-08-02 12:34:13 +03:00
Pekka Enberg
4aea9372bd Rename liblimbo_sqlite3 to libturso_sqlite3 2025-08-02 11:24:40 +03:00
Pekka Enberg
2c05a3e787 Merge 'perf/vdbe: remove eager cloning in op_comparison' from Jussi Saurio
Shaves off about 100-200ms of runtime from TPC-H `19.sql`

Closes #2385
2025-08-02 10:01:47 +03:00
Pekka Enberg
598fdade3e core: Fold HeaderRef to pager module 2025-08-02 09:50:25 +03:00
Pekka Enberg
9a82e269cd Merge 'Update cargo-dist to the latest official version' from Hiroaki Yutani
https://github.com/tursodatabase/turso/pull/1256 switched cargo-dist to
Astral's forked version, but, recently, the official repository got a
new maintainer and started to be maintained again.
Their latest release, [v0.29.0](https://github.com/axodotdev/cargo-
dist/releases/tag/v0.29.0), now includes the features originally added
to Astral's version. So, probably it's a good time to switch back to the
official cargo-dist. That said, as there's no significant changes from
Astral's version, it's also fine to hold the current one.

Closes #2398
2025-08-02 09:46:26 +03:00
Jussi Saurio
43c1afe4b6 Merge 'bindings/rust: Enhance API by removing verbosity' from Diego Reis
While working on #2151 I saw myself forced to do things like:
```rust
assert_eq!(
                6,
                *result
                    .next()
                    .await?
                    .unwrap()
                    .get_value(0)?
                    .as_integer()
                    .unwrap()
            );
```
Just to get a simple value from a row, now with this PR users can just
do:
```rust
assert_eq!(6, result.get::<i32>(0)?);
```
(Thanks libsql devs, this is so much better!)

Closes #2377
2025-08-02 09:39:27 +03:00
Jussi Saurio
c6b178483b Merge 'io_uring: setup plumbing for Fixed opcodes' from Preston Thorpe
This PR by itself is uninteresting and doesn't do anything. But I am
heavily trying to avoid massive PR's, and this is very merge-able
😄

Closes #2396
2025-08-02 09:37:48 +03:00
Jussi Saurio
be1456f7cb Merge 'use state machine for NoConflict opcode' from Mikaël Francoeur
This will save some work when yielding to IO. Previously, on every
invocation, if the record was a packed record, we parsed it and iterated
through the values to check for nulls. Now, the pre-seeking work is done
only once.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2394
2025-08-02 09:37:00 +03:00
Jussi Saurio
37a565021e Merge 'state_machine: remove State associated type' from Pere Diaz Bou
Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2395
2025-08-02 09:36:43 +03:00
Preston Thorpe
97b236f217 Merge 'Fix numeric overflow in numeric string parsing in logical comparision' from
Closes #2077-
This PR fixes an integer overflow bug that causes results from Turso to
differ from SQLite.
-Commit 1: Fixes incorrect logic that failed to detect integer overflow
when parsing long numeric strings.
-Commit 2: Handles the case where a parsed numeric string is stored as a
float and classified as PureInteger, but lies outside the integer range.
Previously, `parsed_value.as_integer()` would return None, causing Turso
to fall back to text comparison against numeric values. This caused
**another** erroneous result, as shown below.
`$> SELECT (-104614899632619 || 45597) > CAST(0 AS NUMERIC); -- tursodb
= 1(wrong), sqlite = 0`
 Now, if Turso fails to convert a very long numeric string to an
integer, it tries to convert it to a float. This is in line with the
`static void applyNumericAffinity` function in sqlite3
**Before**
<img width="623" height="238" alt="Screenshot 2025-08-01 at 12 11 49 PM"
src="https://github.com/user-attachments/assets/796d6ff6-768b-40ef-
ac83-e0c55fff6bd9" />
**After**
`SELECT (104614899632619 || 45597) > CAST(0 AS NUMERIC); -- tursodb = 1,
sqlite = 1`
`SELECT (-104614899632619 || 45597) > CAST(0 AS NUMERIC); -- tursodb =
0, sqlite = 0`

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2397
2025-08-01 21:39:30 -04:00
Levy A.
1e177053cb feat: add RenameTable instruction
direct schema mutation, no reparsing
2025-08-01 21:11:25 -03:00
rajajisai
d2d7adff9e Including test for parsing large numeric strings as number when an operand is numerican when doing logical comparision 2025-08-01 16:30:32 -07:00
Mikaël Francoeur
81412b4a17 use state machine for NoConflict opcode 2025-08-01 17:29:57 -04:00
Hiroaki Yutani
b0de38bbce Update cargo-dist to the latest official version 2025-08-02 04:35:52 +09:00
rajajisai
f6d43df46f Merge branch 'tursodatabase:main' into issue/2077 2025-08-01 15:20:36 -04:00
Diego Reis
31eb4403ad Add integration tests for query_row and get 2025-08-01 16:00:32 -03:00
Diego Reis
8a47b9d5a4 Address PR's comments 2025-08-01 16:00:32 -03:00
Diego Reis
572d3bd4ce Simplify Rust API verbosity by implementing Row.get() and
Statement.query_row()
2025-08-01 16:00:32 -03:00