remote protocol can require client to perform checkpoint of the WAL.
This PR supports that need in the sync-engine implementation
Reviewed-by: Preston Thorpe <preston@turso.tech>
Closes#2565
in #2521, I messed up and introduced improper calculation of the current
checkpoint's max safe frame (mostly due to incorrect comments that I had
left on the method).
The confusion partially stems from our lack of Busy handling at the
moment, but essentially when determining the max safe frame for all
readers, for passive mode we cannot simply `break` out of the loop when
we find a reader with a lower read mark than we have, because _another_
reader might have an even _lower_ read mark, and we could proceed with
the first mark < shared_max.
And for !passive modes, we still attempt to backfill with the same lower
frame, we just return `Busy` at the end, after backfilling what we can
(we just don't reset the log for restart/truncate).
Most of the changes in this PR is just the renaming the fields of
Checkpoint Result, because the names were confusing
Closes#2560
This PR fixes few issues in turso-sync-engine implementation:
1. One step of `pull` implementation works like this:
a. Start write WAL session
b. Revert local changes in WAL
c. Replay WAL frames from remote DB
d. Replay WAL frames produced by local changes applied to the remote
DB copy (`synced`)
My initial thinking was that by executing step (d) we will get the same
schema as before (with same schema cookie) and everything will be fine.
With more deep thinking we can see that it's not fine - as after step
(d) tables created locally can change their root pages (if they were
created locally, for example) - and DB will have "broken" schema
2. Sync engine execute few SQL statements and do not run them to
completion - which basically created "orphaned" locks
In order to fix (1) I decided to introduce another `conn_raw_api`
extension which allows to read and write schema cookie directly in the
transaction. So, the process described above adds step (e) which set
schema cookie to the value strictly greater than the value before.
In order to fix (2) I just fixed all places where statement were dropped
before running to completion.
These fixes are merged together because I explored them by fixing one
new test: `test_sync_single_db_many_pulls_big_payloads`
Closes#2561
