mirror of
https://github.com/aljazceru/turso.git
synced 2025-12-25 03:54:21 +01:00
d0f57584ad
This PR introduces initial protocol for bidirectional sync with conflict resolution. The main addition to the usual `Database` interface are two methods: 1. `push` - push all local changes to the remote. Note, that new changes from the remote will not be visible during request execution after this procedure. 2. `pull` - pull all remote changes and apply them locally. Note, that this procedure require temporary block of writes to the local DB as internally we will manipulate with opened connections and juggle with few things under the hood. ## Limitations * Current implementation exposes only query methods on top of the **database** - because more careful orchestration will be needed when we will expose `Connection` and `Statement` * Schema changes are possible to make through synced Database - but they are actually not synced to the remote * Current implementation will amplify storage use by 2x ## Implementation overview Current approach uses pretty stupid idea to maintain 2 copies of the database and WAL files: 1. `Draft` - this copy will hold local changes and accept all writes made to the database 2. `Synced` - this copy will hold DB file and WAL synced with remote This obviously lead to 2x space amplification, but allow us to implement sync with conflict resolution without changing `turso-core`. Under the hood, implementation of main operations looks like this: 1. `push`: a. Pull all recent changes from the remote to `Synced` DB b. Transfer local changes from `Draft` to `Synced` with the help of CDC table c. Push new WAL frames from `Synced` DB to remote 2. `pull`: a. Pull all recent changes from the remote to `Synced` DB b. Transfer local changes from `Draft` to `Synced` with the help of CDC table c. Copy `Synced` files (DB and WAL) to the `Draft` d. Reset `Synced` WAL in order to remove frames made by local changes from it As operation 2.c can't be made safely without extra work - `turso-sync` package internally maintains `active` database which can be either `Draft` or `Synced` and switch will happen exactly before/after step 2.c as we will need to move all requests from `Draft` DB to `Synced` due to explicit copy which we will need to perform. This switch between Databases creates additional troubles and that's why in this PR only `Database::query` and `Database::execute` methods are exposed without prepared statements. <img width="2062" height="977" alt="Untitled-2025-07-14-1259" src="https://github.com/user- attachments/assets/64eb5046-d7cb-4af2-87a0-810c0db7eeb5" /> <img width="2062" height="977" alt="Untitled-2025-07-14-1259(1)" src="https://github.com/user- attachments/assets/5c20360c-41db-4100-b0ff-9e47c2682e56" /> Closes #2334