aljaz/turso
1
0
Fork 0
mirror of https://github.com/aljazceru/turso.git synced 2026-01-18 15:44:20 +01:00
Code Issues Packages Projects Releases Wiki Activity
112 Commits 1 Branch 0 Tags
34d21b0eb7601b278361fcef8aaf6cfb488bd668
Commit Graph

112 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Piotr Sarna
34d21b0eb7 storage: add S3 storage 
It's now possible to run with S3 persistent storage
if bindings are built with `s3_storage` feature.
Regular S3 credentials need to be set up locally,
and all customary env variables like AWS_SECRET_ACCESS_KEY
and AWS_ACCESS_KEY_ID work just fine.

For local development, one can set MVCCRS_ENDPOINT env variable.
For testing with MinIO, the following setup customarily works:
MVCCRS_ENDPOINT=http://localhost:9000 \
  AWS_SECRET_ACCESS_KEY=minioadmin \
  AWS_ACCESS_KEY_ID=minioadmin \
    ./libsql /tmp/testme.db
 2023-05-16 14:36:58 +02:00
Piotr Sarna
5da87739fa bindings: split transcation begin from insert/read 
libSQL expects to be able to begin/commit a transaction
independently of reading or inserting data.
 2023-05-15 14:33:25 +02:00
Pekka Enberg
57568981a4 Merge pull request #43 from penberg/detrait_storage 
treewide: drop storage trait
 2023-05-15 12:29:55 +03:00
Piotr Sarna
8b1ef20c08 treewide: drop storage trait 
We're good with an enum, and async_trait has a runtime cost
we don't like.
 2023-05-15 10:50:47 +02:00
Pekka Enberg
7772c65f8d Switch to Tokio's mutex (#40) 
The async trait wrapper is very expensive because it's doing pin boxing
in the hot path. Switch to Tokio's mutex to get back performance.

Fixes #39
 2023-05-12 19:25:08 +02:00
Pekka Enberg
df42771c63 Merge pull request #38 from penberg/tableid 
mvcc: switch to (table_id, row_id) for row ids
 2023-05-12 14:20:16 +03:00
Piotr Sarna
3236421f77 mvcc: switch to (table_id, row_id) for row ids 
With that, we're able to easily distinguish rows from different
tables.
 2023-05-12 12:15:19 +02:00
Piotr Sarna
a782ae5a0a cursor: add is_empty 2023-05-12 10:09:18 +02:00
Piotr Sarna
d5eec5d528 cursor: add MVCCScanCursorPosition 2023-05-11 14:11:10 +02:00
Piotr Sarna
582bf14934 database: keep rows in BTreeMap 
We need ordering libSQL-side.
 2023-05-11 13:38:38 +02:00
Piotr Sarna
e8bdfc8e7a cursor, read: update pointers to *mut u8 2023-05-11 13:35:19 +02:00
Piotr Sarna
54ee330912 cursor: add closing cursor 
... which also terminates the read transaction open for scanning.
 2023-05-11 12:13:27 +02:00
Piotr Sarna
5bdcfc9924 cursor: handle current() graciously when there's no data 2023-05-11 11:08:43 +02:00
Pekka Enberg
fb5217959f Merge pull request #37 from penberg/scan_cursor 
mvcc, bindings: expose a scan cursor
 2023-05-10 18:36:55 +03:00
Piotr Sarna
ef097362ff mvcc, bindings: expose a scan cursor 2023-05-10 15:48:46 +02:00
Pekka Enberg
57b5919637 Merge pull request #35 from penberg/recovery 
Initial support for recovery
 2023-05-10 13:28:16 +03:00
Pekka Enberg
0cf7b787fd Merge pull request #36 from penberg/readimpl 
bindings: expose reading from the database
 2023-05-10 13:22:57 +03:00
Piotr Sarna
d047a24a32 bindings: expose reading from the database 
The results are returned as a CString for now.
 2023-05-10 12:09:06 +02:00
Pekka Enberg
08adf2118c Rename mutations.png to transactions.png 2023-05-10 13:01:33 +03:00
Pekka Enberg
ec336d5578 Initial support for recovery 
We have the transaction log in persistent storage so let's implement
recovery by replaying that.
 2023-05-10 12:56:19 +03:00
Pekka Enberg
55bee1dcae Merge pull request #34 from penberg/log-records 
Rename mutations to log records
 2023-05-10 12:18:50 +03:00
Pekka Enberg
8c56b381c0 Rename mutations to log records 
The Hekaton paper talks about "log records" so let's just run with that
terminology to avoid confusion.
 2023-05-09 21:32:36 +03:00
Pekka Enberg
3944a4e987 Merge pull request #31 from penberg/fix-insert 
Fix MVCCDatabaseInsert() type signature
 2023-05-09 16:15:10 +03:00
Pekka Enberg
cf31de5d41 Fix MVCCDatabaseInsert() type signature 
Values are opaque blobs so use "const void *" for C callers.
 2023-05-09 16:14:28 +03:00
Pekka Enberg
a3ac3272eb Merge pull request #29 from penberg/stdint 
Add stdint.h include to mvcc.h
 2023-05-09 13:54:45 +03:00
Pekka Enberg
3d0c8a415e Add stdint.h include to mvcc.h 2023-05-09 13:49:31 +03:00
Pekka Enberg
86fa4c5662 Merge pull request #28 from penberg/staticlib 
mvcc: build static library for C bindings
 2023-05-09 11:30:52 +03:00
Piotr Sarna
d4da54b10b mvcc: build static library for C bindings 
That's how it's currently consumed by libSQL.
 2023-05-09 10:28:26 +02:00
Pekka Enberg
f47eea1a72 Move DESIGN.md to docs 2023-05-09 11:00:00 +03:00
Pekka Enberg
19ea5c287b Merge pull request #27 from penberg/core 
Rename database directory to mvcc-rs
 2023-05-09 10:59:19 +03:00
Pekka Enberg
b9575c4375 Rename database directory to mvcc-rs 2023-05-09 10:56:46 +03:00
Pekka Enberg
cc4db340ee Merge pull request #26 from penberg/c-bindings 
Improve C bindings
 2023-05-09 10:51:39 +03:00
Pekka Enberg
41bed41544 Improve logging 2023-05-09 10:51:14 +03:00
Pekka Enberg
5ce2bc41f9 cargo fmt 2023-05-09 10:47:43 +03:00
Pekka Enberg
b2f46e156b Improve C binding error reporting 2023-05-09 10:47:40 +03:00
Pekka Enberg
3ecb0fb2a9 Mark MVCCDatabaseOpen() as unsafe 2023-05-09 10:44:06 +03:00
Pekka Enberg
db1e313aca Silence clippy 2023-05-09 10:43:42 +03:00
Pekka Enberg
1fc99181f1 Update README.md 2023-05-09 10:42:05 +03:00
Pekka Enberg
c94561a646 Add mvcc.h to the tree 2023-05-09 10:41:33 +03:00
Pekka Enberg
779ad3066a Add error codes to C bindings 2023-05-09 10:41:13 +03:00
Pekka Enberg
34a4f1a269 Improve generated C bindings 
Before:

```c
typedef LocalClock Clock;

typedef JsonOnDisk Storage;

typedef DatabaseInner<Clock, Storage> Inner;

typedef Database<Clock, Storage, Mutex<Inner>> Db;

typedef struct {
  Db db;
  Runtime runtime;
} DbContext;

extern "C" {

DbContext *mvccrs_new_database(const char *path);

void mvccrs_free_database(Db *db);

int32_t mvccrs_insert(DbContext *db, uint64_t id, const uint8_t *value_ptr, uintptr_t value_len);

} // extern "C"

```

After:

```c

typedef struct DbContext DbContext;

typedef const DbContext *MVCCDatabaseRef;

extern "C" {

MVCCDatabaseRef MVCCDatabaseOpen(const char *path);

void MVCCDatabaseClose(MVCCDatabaseRef db);

int32_t MVCCDatabaseInsert(MVCCDatabaseRef db, uint64_t id, const uint8_t *value_ptr, uintptr_t value_len);

} // extern "C"

```
 2023-05-09 10:34:11 +03:00
Pekka Enberg
124446f17c Generate C header file on cargo build 2023-05-09 10:34:11 +03:00
Pekka Enberg
d5b96d5edf Move C bindings to separate crate 2023-05-09 10:34:11 +03:00
Pekka Enberg
69cee068ce Merge pull request #25 from penberg/mvcc_write_cursor 
add c_bindings feature for exposing functions to C
 2023-05-08 15:59:07 +03:00
Piotr Sarna
488201cab2 add c_bindings feature for exposing functions to C 
The library can now be compiled to a static or dynamic
native lib, with very basic functionality exposed.
 2023-05-08 14:20:34 +02:00
Pekka Enberg
aac835fce9 Document persistent storage design (#24) 2023-05-06 20:15:08 +02:00
Pekka Enberg
eb52f3e1cd Merge pull request #23 from psarna/compact_rowversions 2023-04-25 18:56:44 +03:00
Piotr Sarna
fb6ce70993 database: add dropping unused row versions 
When a row version is not visible by any transactions,
active or future ones, it should be dropped.
 2023-04-20 15:47:36 +02:00
Piotr Sarna
2a018ea9a3 fixup: move DatabaseError under a feature 2023-04-20 15:47:36 +02:00
Pekka Enberg
851871d24b Merge pull request #22 from psarna/storfixup 
fixup: implement Stream for JsonOnDiskStream under a feature
 2023-04-20 16:03:51 +03:00