aljaz/turso
1
0
Fork 0
mirror of https://github.com/aljazceru/turso.git synced 2026-01-12 20:54:21 +01:00
Code Issues Packages Projects Releases Wiki Activity
132 Commits 1 Branch 0 Tags
57249f2c94361fefda30336f7d5d98e2d64ccff2
Commit Graph

132 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Piotr Sarna
57249f2c94 concurrency test: port to OS threads 
Without mutexes, it makes no sense anymore to use shuttle.
Instead, the test cases just spawn OS threads.
Also, a case with overlapping ids is added, to test whether
transactions read their own writes within the same transaction.
 2023-06-12 14:27:40 +02:00
Piotr Sarna
a93fcdcbcf database: make transaction state atomic 
Without atomic access, we're subject to races when inspecting
whether a transaction just changed its state, e.g. from Preparing
to Committed.
 2023-06-12 13:14:46 +02:00
Pekka Enberg
3d804ece2a Merge pull request #52 from penberg/lockfree 
database: drop the mutex
 2023-06-12 13:09:12 +03:00
Piotr Sarna
983544dbfd database: implement missing cases for is_version_visible + tests 
Following the Hekaton paper tables, but also taking into account
that in iteration 0 we're only interested in snapshot isolation,
not serializability.
 2023-06-07 13:58:34 +02:00
Piotr Sarna
b4932340f4 database: add a juicy comment about serializability 
And specifically, the amount of things we don't have implemented
to even think of that. It's mostly about tracking commit dependencies
which allow speculative reads/ignores of certain versions,
as well as making sure that in the commit phase, we validate
visibility of all versions read, as well as that our scans
took into account all data. If some version appeared after the transaction
began, and it was not taken into account during its scans, it is considered
a "phantom", and it invalidates the transaction if we strive for
serializability.
 2023-06-07 11:20:00 +02:00
Piotr Sarna
6d82973359 database: restore a CRUCIAL comment about dropping a tx 
... which stops being correct after lock-free!
 2023-06-06 16:04:28 +02:00
Piotr Sarna
ddbcd9be79 database: bring back dropping unused row versions 2023-06-06 12:47:40 +02:00
Piotr Sarna
625394000e unignore test_dirty_read_deleted 2023-06-06 11:34:34 +02:00
Piotr Sarna
47eb149214 database: drop the mutex 
Without a critical section, we naturally hit a few unimplemented
paths when handling concurrent transactions, which is great news!
Visiting previously impossible paths already proves that lock-free
is able to handle concurrency > 1.
Now, the easy part - fixing all the unimplemented paths and making
the Hekaton implementation 100% foolproof.
 2023-06-06 11:28:27 +02:00
Piotr Sarna
57b2e031fb Merge pull request #51 from penberg/moreskipmap 
database: migrate RefCell<HashMap> and RefCell<BTreeMap> to SkipMap
 2023-06-06 09:58:37 +02:00
Piotr Sarna
a8faffa9f5 database: migrate txs to SkipMap 2023-06-06 09:43:00 +02:00
Piotr Sarna
fdfc4fd5b4 database: drop RefCell from SkipMap 
not needed, the structure is already Send&Sync
 2023-06-05 13:58:24 +02:00
Piotr Sarna
74a7628a0a mvcc-rs: move database tests to a separate file 
That makes the file more human-readable
 2023-06-05 12:50:35 +02:00
Piotr Sarna
77e88d3f04 fix clippy 2023-06-05 11:47:53 +02:00
Pekka Enberg
51f33919d3 Switch to concurrent SkipMap for row versions (#49) 
Let's switch to concurrent SkipMap as the first small step towards
lockless index...
 2023-05-17 15:31:16 +02:00
Pekka Enberg
3b9e235d09 Merge pull request #47 from penberg/resync 
treewide: overhaul the API to be sync again
 2023-05-17 13:59:46 +03:00
Piotr Sarna
ae4cc872b6 treewide: overhaul the API to be sync again 
We dropped all occurrences of Tokio to avoid the cost of allocations
induced by async runtimes.
The only async part of the code is now S3 storage, which is just
wrapped in a futures::executor::block_on()
 2023-05-17 12:39:08 +02:00
Pekka Enberg
09a16d33f3 Merge pull request #45 from penberg/s3stor 
storage: add S3 storage
 2023-05-16 16:59:24 +03:00
Piotr Sarna
a28c41f919 bindings: run recovery on MVCCDatabaseOpen 2023-05-16 14:55:16 +02:00
Piotr Sarna
34d21b0eb7 storage: add S3 storage 
It's now possible to run with S3 persistent storage
if bindings are built with `s3_storage` feature.
Regular S3 credentials need to be set up locally,
and all customary env variables like AWS_SECRET_ACCESS_KEY
and AWS_ACCESS_KEY_ID work just fine.

For local development, one can set MVCCRS_ENDPOINT env variable.
For testing with MinIO, the following setup customarily works:
MVCCRS_ENDPOINT=http://localhost:9000 \
  AWS_SECRET_ACCESS_KEY=minioadmin \
  AWS_ACCESS_KEY_ID=minioadmin \
    ./libsql /tmp/testme.db
 2023-05-16 14:36:58 +02:00
Pekka Enberg
d34f19fa9c Merge pull request #44 from penberg/splittx 
bindings: split transcation begin from insert/read
 2023-05-15 15:48:34 +03:00
Piotr Sarna
5da87739fa bindings: split transcation begin from insert/read 
libSQL expects to be able to begin/commit a transaction
independently of reading or inserting data.
 2023-05-15 14:33:25 +02:00
Pekka Enberg
57568981a4 Merge pull request #43 from penberg/detrait_storage 
treewide: drop storage trait
 2023-05-15 12:29:55 +03:00
Piotr Sarna
8b1ef20c08 treewide: drop storage trait 
We're good with an enum, and async_trait has a runtime cost
we don't like.
 2023-05-15 10:50:47 +02:00
Pekka Enberg
7772c65f8d Switch to Tokio's mutex (#40) 
The async trait wrapper is very expensive because it's doing pin boxing
in the hot path. Switch to Tokio's mutex to get back performance.

Fixes #39
 2023-05-12 19:25:08 +02:00
Pekka Enberg
df42771c63 Merge pull request #38 from penberg/tableid 
mvcc: switch to (table_id, row_id) for row ids
 2023-05-12 14:20:16 +03:00
Piotr Sarna
3236421f77 mvcc: switch to (table_id, row_id) for row ids 
With that, we're able to easily distinguish rows from different
tables.
 2023-05-12 12:15:19 +02:00
Piotr Sarna
a782ae5a0a cursor: add is_empty 2023-05-12 10:09:18 +02:00
Piotr Sarna
d5eec5d528 cursor: add MVCCScanCursorPosition 2023-05-11 14:11:10 +02:00
Piotr Sarna
582bf14934 database: keep rows in BTreeMap 
We need ordering libSQL-side.
 2023-05-11 13:38:38 +02:00
Piotr Sarna
e8bdfc8e7a cursor, read: update pointers to *mut u8 2023-05-11 13:35:19 +02:00
Piotr Sarna
54ee330912 cursor: add closing cursor 
... which also terminates the read transaction open for scanning.
 2023-05-11 12:13:27 +02:00
Piotr Sarna
5bdcfc9924 cursor: handle current() graciously when there's no data 2023-05-11 11:08:43 +02:00
Pekka Enberg
fb5217959f Merge pull request #37 from penberg/scan_cursor 
mvcc, bindings: expose a scan cursor
 2023-05-10 18:36:55 +03:00
Piotr Sarna
ef097362ff mvcc, bindings: expose a scan cursor 2023-05-10 15:48:46 +02:00
Pekka Enberg
57b5919637 Merge pull request #35 from penberg/recovery 
Initial support for recovery
 2023-05-10 13:28:16 +03:00
Pekka Enberg
0cf7b787fd Merge pull request #36 from penberg/readimpl 
bindings: expose reading from the database
 2023-05-10 13:22:57 +03:00
Piotr Sarna
d047a24a32 bindings: expose reading from the database 
The results are returned as a CString for now.
 2023-05-10 12:09:06 +02:00
Pekka Enberg
08adf2118c Rename mutations.png to transactions.png 2023-05-10 13:01:33 +03:00
Pekka Enberg
ec336d5578 Initial support for recovery 
We have the transaction log in persistent storage so let's implement
recovery by replaying that.
 2023-05-10 12:56:19 +03:00
Pekka Enberg
55bee1dcae Merge pull request #34 from penberg/log-records 
Rename mutations to log records
 2023-05-10 12:18:50 +03:00
Pekka Enberg
8c56b381c0 Rename mutations to log records 
The Hekaton paper talks about "log records" so let's just run with that
terminology to avoid confusion.
 2023-05-09 21:32:36 +03:00
Pekka Enberg
3944a4e987 Merge pull request #31 from penberg/fix-insert 
Fix MVCCDatabaseInsert() type signature
 2023-05-09 16:15:10 +03:00
Pekka Enberg
cf31de5d41 Fix MVCCDatabaseInsert() type signature 
Values are opaque blobs so use "const void *" for C callers.
 2023-05-09 16:14:28 +03:00
Pekka Enberg
a3ac3272eb Merge pull request #29 from penberg/stdint 
Add stdint.h include to mvcc.h
 2023-05-09 13:54:45 +03:00
Pekka Enberg
3d0c8a415e Add stdint.h include to mvcc.h 2023-05-09 13:49:31 +03:00
Pekka Enberg
86fa4c5662 Merge pull request #28 from penberg/staticlib 
mvcc: build static library for C bindings
 2023-05-09 11:30:52 +03:00
Piotr Sarna
d4da54b10b mvcc: build static library for C bindings 
That's how it's currently consumed by libSQL.
 2023-05-09 10:28:26 +02:00
Pekka Enberg
f47eea1a72 Move DESIGN.md to docs 2023-05-09 11:00:00 +03:00
Pekka Enberg
19ea5c287b Merge pull request #27 from penberg/core 
Rename database directory to mvcc-rs
 2023-05-09 10:59:19 +03:00