mirror of
https://github.com/aljazceru/turso.git
synced 2025-12-25 20:14:21 +01:00
380b27f58a
I searched using deepwiki how SQLite implements their busy handler. They use a callback system with exponential backoff, where it stores the callback in the pager and in the database. I confess I found this slightly confusing, so I just implemented a simple exponential backoff directly in the `Statement` struct. I imagine SQLite does this in a more convoluted manner, as they do not have a concept of yielding as we do. https://deepwiki.com/search/where-is-the-code-for-the- busy_4a5ed006-4eed-479f-80c3-dd038832831b I also fixed the rust bindings so that it yields when we return `StepResult::IO`, instead of just blocking the async function. To achieve this I implemented the `Stream` trait for `Rows` struct, which unfortunately came with a slight change to the function signature of `rows.next()` to `rows.try_next()`. EDIT: ~test `test_multiple_connections_fuzz` timeouts because now it has the busy handler "slowing" things down (this test generates a lot of busy transactions), so it takes a lot longer for the test to run. Not sure if it is acceptable for us to reduce the number of operations so the test is shorter.~ EDIT: Adjusted the API to be more in line with https://www.sqlite.org/c3ref/busy_timeout.html. Sets maximum total accumulated timeout. If the duration is None or Zero, we unset the busy handler for this Connection. This api defers slightly from SQLite as instead of sleeping for linear amount of time specified by the user, we will sleep in phases until the the total amount of time requested is reached. This means we first sleep of 1ms, then if we still return busy, we sleep for 2 ms, and repeat until a maximum of 100 ms per phase or we reached the total timeout. Example: 1. Set duration to 5ms 2. Step through query -> returns Busy -> sleep/yield for 1 ms 3. Step through query -> returns Busy -> sleep/yield for 2 ms 4. Step through query -> returns Busy -> sleep/yield for 2 ms (totaling 5 ms of sleep) 5. Step through query -> returns Busy -> return Busy to user This slight api change demonstrated a better throughtput in `perf/throughput/turso` benchmark ```sh cargo run -p write-throughput --release -- -t 2 Running write throughput benchmark with 2 threads, 100 batch size, 10 iterations, mode: Legacy Database created at: write_throughput_test.db Thread 1: 1000 inserts in 0.04s (23438.42 inserts/sec) Thread 0: 1000 inserts in 0.08s (12385.64 inserts/sec) === BENCHMARK RESULTS === Total inserts: 2000 Total time: 0.08s Overall throughput: 24762.60 inserts/sec Threads: 2 Batch size: 100 Iterations per thread: 10 Database file exists: true Database file size: 4096 bytes ``` Depends on #3102 Closes #3067 Closes #3074