Commit Graph

7049 Commits

Author SHA1 Message Date
pedrocarlo
e9c3f0d55b disable schema reprepare for MVCC 2025-08-04 12:32:34 -03:00
pedrocarlo
0e3e64878c workaround the fact that to reparse schema we have to avoid falling into a reprepared statement loop 2025-08-04 12:32:34 -03:00
pedrocarlo
266a7e1c66 do not error in op_transaction if page 1 was not allocated 2025-08-04 12:32:34 -03:00
pedrocarlo
f0ff85a43c add test 2025-08-04 12:32:34 -03:00
pedrocarlo
0779c23bbf fix merge conflicts 2025-08-04 12:32:34 -03:00
pedrocarlo
d2019e95f3 pass schema to epilogue for schema_version checking + do not Pragma Schema Version in open_with_flags to avoid infinite loop in reprepare. Just access the database header directly 2025-08-04 12:32:34 -03:00
pedrocarlo
736748cdf7 Simplify program epilogue by tracking the transaction mode and rollback status in the ProgramBuilder and then calling epilogue just once 2025-08-04 12:32:34 -03:00
pedrocarlo
c567636deb Adjust Transaction OpCode to accept schema cookie + check if cookie changed 2025-08-04 12:32:34 -03:00
pedrocarlo
f8eb4ba14d implement reprepare for statements 2025-08-04 12:32:34 -03:00
pedrocarlo
cffafb8333 remove Rc from Program 2025-08-04 12:32:34 -03:00
pedrocarlo
54636241c2 store Sql String inside Program for reprepare 2025-08-04 12:32:34 -03:00
pedrocarlo
94c25893ae add SchemaUpdated error variant 2025-08-04 12:32:34 -03:00
Jussi Saurio
506bb5f67f Merge 'Direct schema mutation – add instruction' from Levy A.
Resolves #2378.
```
`ALTER TABLE _ RENAME TO _`/limbo_rename_table/
                        time:   [15.645 ms 15.741 ms 15.850 ms]
Found 12 outliers among 100 measurements (12.00%)
  8 (8.00%) high mild
  4 (4.00%) high severe
`ALTER TABLE _ RENAME TO _`/sqlite_rename_table/
                        time:   [34.728 ms 35.260 ms 35.955 ms]
Found 15 outliers among 100 measurements (15.00%)
  8 (8.00%) high mild
  7 (7.00%) high severe
  ```
<img width="1000" height="199" alt="image" src="https://github.com/user-
attachments/assets/ad943355-b57d-43d9-8a84-850461b8af41" />

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2399
2025-08-04 16:55:38 +03:00
Jussi Saurio
1813171b91 Merge 'Use pwrite for single buffer pwritev call in unix IO' from Preston Thorpe
Closes #2416
2025-08-04 16:52:14 +03:00
Jussi Saurio
5a06411ce6 Merge 'fix/core/translate: ALTER TABLE DROP COLUMN: ensure schema cookie is updated even when target table is empty' from Jussi Saurio
Closes #2431
Discovered while fuzzing #2086
## What
We update `schema_version` whenever the schema changes
## Problem
Probably unintentionally, we were calling `SetCookie` in a loop for each
row in the target table, instead of only once at the end. This means 2
things:
- For large `n`, this is a lot of unnecessary instructions
- For `n==0`, `SetCookie` doesn't get called at all -> the schema won't
get marked as having been updated -> conns can operate on a stale schema
## Fix
Lift `SetCookie` out of the loop

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2432
2025-08-04 16:51:24 +03:00
Jussi Saurio
8a1723b3c8 fix/core/translate: ALTER TABLE DROP COLUMN: ensure schema cookie is updated even when target table is empty 2025-08-04 15:05:00 +03:00
Pekka Enberg
e4accdc29d Merge 'hide dangerous methods behind conn_raw_api feature' from Nikita Sivukhin
WAL API shouldn't be exposed by default because this is relatively
dangerous API which we use internally and ordinary users shouldn't not
be interested in it.

Reviewed-by: Pekka Enberg <penberg@iki.fi>

Closes #2424
2025-08-04 14:52:40 +03:00
Pekka Enberg
1572285ee6 Merge 'preserve files in IO memory backend' from Nikita Sivukhin
Simple PR to preserve and reuse files in memory IO

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2428
2025-08-04 14:52:24 +03:00
Pekka Enberg
5b5bc441b7 Merge ' core/mvcc: fix new rowid on restart' from Pere Diaz Bou
Next rowid was being tracked globally for all tables and restarted to 0
every time database was opened

Closes #2425
2025-08-04 14:37:13 +03:00
Nikita Sivukhin
129895f0b2 preserve files in IO memory backend 2025-08-04 15:22:04 +04:00
Pere Diaz Bou
56240ddac9 core/mvcc: add restart tests 2025-08-04 12:31:17 +02:00
Pere Diaz Bou
f26e442597 core/mvcc: fix new rowid
next rowid was being tracked globally for all tables and restarted to 0
every time database was opened
2025-08-04 12:31:17 +02:00
Pere Diaz Bou
83a658d3d6 core/mvcc: add option to test with a random file and restart it 2025-08-04 12:31:17 +02:00
Nikita Sivukhin
83b1e99a61 fix compilation 2025-08-04 12:53:07 +04:00
Nikita Sivukhin
0adb40534c hind dangerous methods behind conn_raw_api feature 2025-08-04 12:40:28 +04:00
Pekka Enberg
deec70e541 Merge 'Improve SQLite3 TCL test suite' from Pekka Enberg
Add more stubs in tester.tcl so that the test suite does not bail out
early.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2405
2025-08-04 08:43:14 +03:00
Pekka Enberg
ca14799da5 Merge 'Make completions idempotent' from Preston Thorpe
Closes #2417
2025-08-04 08:42:42 +03:00
Pekka Enberg
3691b51039 Merge 'perf/btree: skip seek in move_to_rightmost() if we are already on rightmost page' from Jussi Saurio
## Background
When we get a new rowid using `op_new_rowid()`, we move to the end of
the btree to look at what the maximum rowid currently is, and then
increment it by one.
This requires a btree seek.
## Problem
If we were already on the rightmost page, this is a lot of unnecessary
work, including potentially a few page reads from disk (although to be
fair the ancestor pages are very likely to be in cache at this point.)
## Fix
Cache the rightmost page id whenever we enter it in
`move_to_rightmost()`, and invalidate it whenever we do a balancing
operation.
## Local benchmark results
```sql
Insert rows in batches/limbo_insert_1_rows
                        time:   [23.333 µs 27.718 µs 35.801 µs]
                        change: [-7.7924% +0.8805% +12.841%] (p = 0.91 > 0.05)
                        No change in performance detected.
Insert rows in batches/limbo_insert_10_rows
                        time:   [38.204 µs 38.381 µs 38.568 µs]
                        change: [-8.7188% -7.4786% -6.1955%] (p = 0.00 < 0.05)
                        Performance has improved.
Insert rows in batches/limbo_insert_100_rows
                        time:   [158.39 µs 165.06 µs 178.37 µs]
                        change: [-21.000% -18.789% -15.666%] (p = 0.00 < 0.05)
                        Performance has improved.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2409
2025-08-04 08:41:51 +03:00
PThorpe92
79629daff4 Make completions idempotent 2025-08-02 21:48:39 -04:00
Levy A.
b9a3a93ef0 fix: clippy 2025-08-02 20:06:05 -03:00
PThorpe92
b5117ac5c7 Use pwrite for single buffer in unix IO 2025-08-02 18:34:16 -04:00
Levy A.
b14a11a2fd fix: change name for schema btree + fix benchmark 2025-08-02 17:17:36 -03:00
Jussi Saurio
130e1f80ea fix/vdbe: call seek_to_last() only once in op_new_rowid 2025-08-02 14:18:58 +03:00
Jussi Saurio
63a5ef596b perf/btree: skip seek in move_to_rightmost() if we are already on rightmost page 2025-08-02 13:56:59 +03:00
Pekka Enberg
a4dcbeb392 Update PERF.md 2025-08-02 13:37:42 +03:00
Jussi Saurio
3b0c8b08fe Merge 'perf/pager: dont clear page cache on commit' from Jussi Saurio
This should be safe to do as:
1. page cache is private per connection
2. since this connection wrote the flushed pages/frames, they are up to
date from its perspective
3. multiple concurrent statements inside one connection are not
snapshot-transactional even in sqlite

Reviewed-by: Pekka Enberg <penberg@iki.fi>

Closes #2407
2025-08-02 13:35:57 +03:00
Jussi Saurio
4497d22d3f perf/pager: dont clear page cache on commit 2025-08-02 13:09:36 +03:00
Pekka Enberg
5037e2dc0d testing/sqlite3: Update all.test 2025-08-02 13:00:18 +03:00
Pekka Enberg
779b8e0149 testing/sqlite3: Import more join test cases 2025-08-02 12:59:11 +03:00
Pekka Enberg
c7497d55b6 testing/sqlite3: Disable select9.test
...it seems to run forever.
2025-08-02 12:59:11 +03:00
Pekka Enberg
a380e0775d testing/sqlite3: Improve tester.tcl
Improve tester.tcl by adding stubs so that we can run more of the test
suite without the test harness giving up.
2025-08-02 12:59:11 +03:00
Pekka Enberg
bca9994cf6 Merge 'Rename liblimbo_sqlite3 to libturso_sqlite3' from Pekka Enberg
Closes #2403
2025-08-02 12:34:19 +03:00
Pekka Enberg
12455c6531 Merge 'core: Fold HeaderRef to pager module' from Pekka Enberg
Closes #2401
2025-08-02 12:34:13 +03:00
Pekka Enberg
4aea9372bd Rename liblimbo_sqlite3 to libturso_sqlite3 2025-08-02 11:24:40 +03:00
Pekka Enberg
2c05a3e787 Merge 'perf/vdbe: remove eager cloning in op_comparison' from Jussi Saurio
Shaves off about 100-200ms of runtime from TPC-H `19.sql`

Closes #2385
2025-08-02 10:01:47 +03:00
Pekka Enberg
598fdade3e core: Fold HeaderRef to pager module 2025-08-02 09:50:25 +03:00
Pekka Enberg
9a82e269cd Merge 'Update cargo-dist to the latest official version' from Hiroaki Yutani
https://github.com/tursodatabase/turso/pull/1256 switched cargo-dist to
Astral's forked version, but, recently, the official repository got a
new maintainer and started to be maintained again.
Their latest release, [v0.29.0](https://github.com/axodotdev/cargo-
dist/releases/tag/v0.29.0), now includes the features originally added
to Astral's version. So, probably it's a good time to switch back to the
official cargo-dist. That said, as there's no significant changes from
Astral's version, it's also fine to hold the current one.

Closes #2398
2025-08-02 09:46:26 +03:00
Jussi Saurio
43c1afe4b6 Merge 'bindings/rust: Enhance API by removing verbosity' from Diego Reis
While working on #2151 I saw myself forced to do things like:
```rust
assert_eq!(
                6,
                *result
                    .next()
                    .await?
                    .unwrap()
                    .get_value(0)?
                    .as_integer()
                    .unwrap()
            );
```
Just to get a simple value from a row, now with this PR users can just
do:
```rust
assert_eq!(6, result.get::<i32>(0)?);
```
(Thanks libsql devs, this is so much better!)

Closes #2377
2025-08-02 09:39:27 +03:00
Jussi Saurio
c6b178483b Merge 'io_uring: setup plumbing for Fixed opcodes' from Preston Thorpe
This PR by itself is uninteresting and doesn't do anything. But I am
heavily trying to avoid massive PR's, and this is very merge-able
😄

Closes #2396
2025-08-02 09:37:48 +03:00
Jussi Saurio
be1456f7cb Merge 'use state machine for NoConflict opcode' from Mikaël Francoeur
This will save some work when yielding to IO. Previously, on every
invocation, if the record was a packed record, we parsed it and iterated
through the values to check for nulls. Now, the pre-seeking work is done
only once.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2394
2025-08-02 09:37:00 +03:00