essentially after the first runthrough of `op_transaction` per a given
`ProgramState`, we weren't resetting the instruction state to `Start´ at
all, which means we didn't do any transaction state checking/updating
after that.
PR includes a rust bindings regression test that used to panic before
this change, and I bet it also fixes this issue in turso-go:
https://github.com/tursodatabase/turso-go/issues/28Closes#3516
essentially after the first runthrough of `op_transaction` per a
given `ProgramState`, we weren't resetting the instruction state
to `Start´ at all, which means we didn't do any transaction state
checking/updating.
PR includes a rust bindings regression test that used to panic before
this change, and I bet it also fixes this issue in turso-go:
https://github.com/tursodatabase/turso-go/issues/28
Now returns e.g.:
```rust
SqlExecutionFailure(
"Invalid argument supplied: Unknown parameter ':email' for query 'INSERT INTO users (email, created_at) VALUES (?, ?)'.
Make sure you're using the correct parameter syntax - named: (:foo), positional: (?, ?)"
)
```
instead of unwrapping a None value and panicing
Closes#3515
Now returns e.g.:
```rust
SqlExecutionFailure(
"Invalid argument supplied: Unknown parameter ':email' for query 'INSERT INTO users (email, created_at) VALUES (?, ?)'.
Make sure you're using the correct parameter syntax - named: (:foo), positional: (?, ?)"
)
```
instead of unwrapping a None value and panicing
Closes#3470
## Background
In a query like `SELECT * FROM t LEFT JOIN s ON t.a=s.a WHERE s.a =
'foo'` we can remove the LEFT JOIN and replace it with an `INNER JOIN`
because NULL values will never be equal to 'foo'. Rewriting as `INNER
JOIN` allows the optimizer to also reorder the table join order to come
up with a more efficient query plan. In fact, we have this optimization
already.
## Problem
However, there is a dumb bug where `WhereTerm`s involving this join
still retain their `from_outer_join` state, resulting in forcing the
evaluation of those terms at the original join index, which results in
completely wrong bytecode if the join optimizer decides to reorder the
join as `s JOIN t` instead. Effectively it will evaluate `t.a=s.a` after
table `s` is open but table `t` is not open yet.
## Fix
This PR fixes that issue by clearing `from_outer_join` properly from the
relevant `WhereTerm`s.
Closes#3475
Fixes:
- `start_value` and `length_value` should be casted to integers
- proper handling of utf-8 characters
- do not need to cast blob to string, as substr in blobs refers to byte
indexes and not char-indexes
Closes#3465
Before, FROM clauses were simply ignored:
```
turso> update t set a = b from (select random() as b);
× Parse error: no such column: b
```
Now, they will be rejected with a clear message. It also makes it
clearer that they need to be implemented:
```
turso> update t set a = b from (select random() as b);
× Parse error: FROM clause is not supported in UPDATE
```
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#3509
The page cache implementation uses a pre-allocated vector (`entries`)
with fixed capacity, along with a custom hash map and freelist. This
design requires expensive upfront allocation when creating a new
connection, which severely impacted performance in workloads that open
many short-lived connections (e.g., our concurrent write benchmarks that
create a new connection per transaction).
Therefore, replace the pre-allocated vector with an intrusive doubly-
linked list. This eliminates the page cache initialization overhead from
connection establishment, but also reduces memory usage to entries that
are actually used. Furthermore, the approach allows us to grow the page
cache with much less overhead.
The patch improves concurrent write throughput benchmark by 4x for
single-threaded performance.
Before:
```
$ write-throughput --threads 1 --batch-size 100 -i 1000 --mode concurrent
Running write throughput benchmark with 1 threads, 100 batch size, 1000 iterations, mode: Concurrent
Database created at: write_throughput_test.db
Thread 0: 100000 inserts in 3.82s (26173.63 inserts/sec)
```
After:
```
$ write-throughput --threads 1 --batch-size 100 -i 1000 --mode concurrent
Running write throughput benchmark with 1 threads, 100 batch size, 1000 iterations, mode: Concurrent
Database created at: write_throughput_test.db
Thread 0: 100000 inserts in 0.90s (110848.46 inserts/sec)
```
Closes#3456
- mostly needed for Drizzle - because other clients with ESM can just
use await connect(...) wrapper
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#3462
The page cache implementation uses a pre-allocated vector (`entries`)
with fixed capacity, along with a custom hash map and freelist. This
design requires expensive upfront allocation when creating a new
connection, which severely impacted performance in workloads that open
many short-lived connections (e.g., our concurrent write benchmarks that
create a new connection per transaction).
Therefore, replace the pre-allocated vector with an intrusive
doubly-linked list. This eliminates the page cache initialization
overhead from connection establishment, but also reduces memory usage to
entries that are actually used. Furthermore, the approach allows us to
grow the page cache with much less overhead.
The patch improves concurrent write throughput benchmark by 4x for
single-threaded performance.
Before:
```
$ write-throughput --threads 1 --batch-size 100 -i 1000 --mode concurrent
Running write throughput benchmark with 1 threads, 100 batch size, 1000 iterations, mode: Concurrent
Database created at: write_throughput_test.db
Thread 0: 100000 inserts in 3.82s (26173.63 inserts/sec)
```
After:
```
$ write-throughput --threads 1 --batch-size 100 -i 1000 --mode concurrent
Running write throughput benchmark with 1 threads, 100 batch size, 1000 iterations, mode: Concurrent
Database created at: write_throughput_test.db
Thread 0: 100000 inserts in 0.90s (110848.46 inserts/sec)
```
Replace #[tokio::main] with explicit Runtime builder to set the number
of tokio worker threads to match the benchmark thread count. This
ensures proper thread control and avoids interference from default
tokio thread pool sizing.
not sure how these would even work with mvcc - either way, an ephemeral
table use an ephemeral database file and pager so i don't think putting
its writes into MV store makes sense
TBH i have no idea if there are any weird interactions here but the code
we have now for sure does not work
Closes#3486
Reviewed-by: Nikita Sivukhin (@sivukhin)
Closes#3490
Depends on #3272.
First big step towards: #1851
- Add ignore error flag to `Interaction` to ignore parse errors when
needed, and still properly report other errors from intermediate
queries.
- adjusted shrinking to accommodate transaction statements from
different connections and properly remove extensional queries from some
properties
- MVCC: generates `Begin Concurrent` and `Commit` statements that are
interleaved to test snapshot isolation between connection transactions.
- MVCC: if the next interactions are going to contain a DDL statement,
we first commit all transaction and execute the DDL statements serially
Closes#3278
Remove verbose output from rusqlite benchmark and output only CSV
format: system,threads,batch_size,compute,throughput
This makes it easier to parse and plot benchmark results.
Replace the sleep-based --think parameter with a --compute parameter
that uses a busy loop to simulate realistic CPU or GPU bound business
logic (e.g., parsing, data aggregation, or ML inference). The compute
time is now specified in microseconds instead of milliseconds for
finer granularity.
