In insert_version_raw(), we correctly iterate the versions backwards
because we want to find the newest version that is still older than
the one we are inserting.
However, the order of `.enumerate()` and `.rev()` was wrong, so the
insertion position was calculated based on the position in the
_reversed_ iterator, not the original iterator.
Blacksmith runners have a lot of variance in performance, making it hard
for Nyrkiö to do its job. Discussed on [Discord](https://discord.com/cha
nnels/1258658826257961020/1402269486752469085)
Reviewed-by: Henrik Ingo <henrik@nyrk.io>
Closes#2448
This removes 4 crates from the `cargo build` and tries to ensure that in
the future we avoid the same crates with different versions.
Reviewed-by: Preston Thorpe <preston@turso.tech>
Closes#3141
current logic can lead to a situation where:
- we call read_page(trunk_page_id)
- we assign trunk_page in the FreePageState state machine
- the page read fails and cache marks it as !locked && !loaded
- next call to Pager::free_page() asserts that the page is loaded and
panics
Whopper takes so long to run that i wasn't patient enough, but i'm
pretty sure this closes#3101
Reviewed-by: Preston Thorpe <preston@turso.tech>
Closes#3139
current logic can lead to a situation where:
- we call read_page(trunk_page_id)
- we assign trunk_page in the FreePageState state machine
- the page read fails and cache marks it as !locked && !loaded
- next call to Pager::free_page() asserts that the page is loaded and panics
We now panic on fsync error by default to be safe against fsyncgate.
However, no reason to do that in the stress tester, especially since we
test out of disk space errors under Antithesis.
Closes#3131
Fixes `write-throughput` benchmark deadlocking on 2 threads or more. The
gist of the PR is in the big code comment:
```rust
// important not to hold shared lock beyond this point to avoid deadlock scenario where:
// thread 1: takes readlock here, passes reference to shared.file to begin_read_wal_frame
// thread 2: tries to acquire write lock elsewhere
// thread 1: tries to re-acquire read lock in the completion (see 'complete' above)
//
// this causes a deadlock due to the locking policy in parking_lot:
// from https://docs.rs/parking_lot/latest/parking_lot/type.RwLock.html:
// "This lock uses a task-fair locking policy which avoids both reader and writer starvation.
// This means that readers trying to acquire the lock will block even if the lock is unlocked
// when there are writers waiting to acquire the lock.
// Because of this, attempts to recursively acquire a read lock within a single thread may result in a deadlock."
```
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#3132
Based on #3126Closes#3029Closes#3030Closes#3065Closes#3083Closes#3084Closes#3085
simple reason why mvcc update didn't work: it didn't try to update.
Closes#3127
This PR fixes incorrect path registration for sync in browser, add tests
and also expose revision string in the `stats()` method of synced
database
Closes#3124
We now panic on fsync error by default to be safe against fsyncgate.
However, no reason to do that in the stress tester, especially since we
test out of disk space errors under Antithesis.
I searched using deepwiki how SQLite implements their busy handler. They
use a callback system with exponential backoff, where it stores the
callback in the pager and in the database. I confess I found this
slightly confusing, so I just implemented a simple exponential backoff
directly in the `Statement` struct. I imagine SQLite does this in a more
convoluted manner, as they do not have a concept of yielding as we do.
https://deepwiki.com/search/where-is-the-code-for-the-
busy_4a5ed006-4eed-479f-80c3-dd038832831b
I also fixed the rust bindings so that it yields when we return
`StepResult::IO`, instead of just blocking the async function. To
achieve this I implemented the `Stream` trait for `Rows` struct, which
unfortunately came with a slight change to the function signature of
`rows.next()` to `rows.try_next()`.
EDIT:
~test `test_multiple_connections_fuzz` timeouts because now it has the
busy handler "slowing" things down (this test generates a lot of busy
transactions), so it takes a lot longer for the test to run. Not sure if
it is acceptable for us to reduce the number of operations so the test
is shorter.~
EDIT:
Adjusted the API to be more in line with
https://www.sqlite.org/c3ref/busy_timeout.html.
Sets maximum total accumulated timeout. If the duration is None or Zero,
we unset the busy handler for this Connection.
This api defers slightly from SQLite as instead of sleeping for linear
amount of time specified by the user, we will sleep in phases until the
the total amount of time requested is reached. This means we first sleep
of 1ms, then if we still return busy, we sleep for 2 ms, and repeat
until a maximum of 100 ms per phase or we reached the total timeout.
Example:
1. Set duration to 5ms
2. Step through query -> returns Busy -> sleep/yield for 1 ms
3. Step through query -> returns Busy -> sleep/yield for 2 ms
4. Step through query -> returns Busy -> sleep/yield for 2 ms (totaling
5 ms of sleep)
5. Step through query -> returns Busy -> return Busy to user
This slight api change demonstrated a better throughtput in
`perf/throughput/turso` benchmark
```sh
cargo run -p write-throughput --release -- -t 2
Running write throughput benchmark with 2 threads, 100 batch size, 10 iterations, mode: Legacy
Database created at: write_throughput_test.db
Thread 1: 1000 inserts in 0.04s (23438.42 inserts/sec)
Thread 0: 1000 inserts in 0.08s (12385.64 inserts/sec)
=== BENCHMARK RESULTS ===
Total inserts: 2000
Total time: 0.08s
Overall throughput: 24762.60 inserts/sec
Threads: 2
Batch size: 100
Iterations per thread: 10
Database file exists: true
Database file size: 4096 bytes
```
Depends on #3102Closes#3067Closes#3074
1. commit state machine was assuming that begin_write_tx() cannot fail,
but it can fail if there is another tx that is not using BEGIN
CONCURRENT.
2. if a brand new non-CONCURRENT transaction attempts to start exclusive
transaction but fails with Busy, we must end the read pager read tx it
just started, because otherwise the next time it attempts to do
something it will panic with: `"cannot start a new read tx without
ending an existing one"`
Closes#3125
1. commit state machine was assuming that begin_write_tx() cannot
fail, but it can fail if there is another tx that is not using
BEGIN CONCURRENT.
2. if a brand new non-CONCURRENT transaction attempts to start
exclusive transaction but fails with Busy, we must end the read
pager read tx it just started, because otherwise the next time
it attempts to do something it will panic with:
"cannot start a new read tx without ending an existing one"
**test/fuzz: introduce fuzzoptions to tx isolation test**
this makes it significantly easier to tweak the tx isolation test
parameters, and also makes it much easier to run the MVCC version of the
test without manually tweaking code inline to make it work.
introduces default options for the non-mvcc and mvcc test variants.
---
**test/fuzz: improve error handling in tx isolation fuzz test**
- extract out common behavior for checking acceptable errors
- add functionality to check which errors require rolling back
a transaction
Closes#3118
