Commit Graph

5506 Commits

Author SHA1 Message Date
pedrocarlo
9aa733f80c sleep inside Io completion 2025-06-26 22:17:28 -03:00
pedrocarlo
bac5e4b563 refactor File and Database Storage to remove Arc<Connection> and return Arc<Connection> for caller to wait for completion 2025-06-26 22:17:28 -03:00
pedrocarlo
64d9193e7b refactor Completion to have a type field and lift common is_complete property 2025-06-26 22:17:27 -03:00
pedrocarlo
e2aafacbb4 add cli option to adjust latency probability 2025-06-26 22:17:27 -03:00
pedrocarlo
c8937976e5 generate_latency for each io operation on the file 2025-06-26 22:17:27 -03:00
pedrocarlo
56547f7127 change refcell values to cell 2025-06-26 22:17:27 -03:00
Pekka Enberg
e162b56d01 Merge 'Rename Limbo to Turso Database' from Pekka Enberg
Closes #1835
2025-06-26 21:35:48 +03:00
Pekka Enberg
662615a7ad antithesis: Fix endpoint name too
Spotted by Eric Dinh from Antithesis.
2025-06-26 21:28:10 +03:00
Pekka Enberg
018b17b6f2 Rename Limbo to Turso Database 2025-06-26 21:05:02 +03:00
Pekka Enberg
b702b4e59c Merge 'bindings/javascript: Fix StepResult:IO handling' from Pekka Enberg
Closes #1841
2025-06-26 20:50:35 +03:00
Pekka Enberg
572c722390 Merge 'write page1 on database initialization' from Pere Diaz Bou
Page 1 must be initialized and written as soon as possible without
marking page as dirty.
OpenEphemeral now requires a state machine to accomodate new
begin_write_tx semantics.

Closes #1839
2025-06-26 20:43:40 +03:00
Pekka Enberg
dc2e7ce942 bindings/javascript: Fix StepResult:IO handling 2025-06-26 20:42:35 +03:00
Pekka Enberg
9620c0cd8b antithesis: Fix experiment duration parameter for launch.sh
The endpoint changed on Antithesis side to avoid the complaint about not
joining a network so let's adjust the script.
2025-06-26 20:28:58 +03:00
Pere Diaz Bou
aa93b70a96 empty -> unitialized 2025-06-26 17:59:23 +02:00
Pere Diaz Bou
e341b80051 clippy 2025-06-26 15:01:54 +02:00
Pere Diaz Bou
4d80b8237d write page1 on database initialization
Page 1 must be initialized and written as soon as possible without
marking page as dirty.
2025-06-26 14:44:23 +02:00
Pekka Enberg
d5b06b96ef Merge 'Bring back TPC-H benchmarks' from Pekka Enberg
Closes #1836
2025-06-26 11:38:05 +03:00
Pekka Enberg
129713018c Bring back TPC-H benchmarks 2025-06-26 10:27:46 +03:00
Pekka Enberg
00edfa888c Merge 'Switch to runtime flag for enabling indexes' from Pekka Enberg
Makes it easier to test the feature:
```
$ cargo run --  --experimental-indexes
Limbo v0.0.22
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database
limbo> CREATE TABLE t(x);
limbo> CREATE INDEX t_idx ON t(x);
limbo> DROP INDEX t_idx;
```

Closes #1831
2025-06-26 10:26:40 +03:00
Pekka Enberg
2fc5c0ce5c Switch to runtime flag for enabling indexes
Makes it easier to test the feature:

```
$ cargo run --  --experimental-indexes
Limbo v0.0.22
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database
limbo> CREATE TABLE t(x);
limbo> CREATE INDEX t_idx ON t(x);
limbo> DROP INDEX t_idx;
```
2025-06-26 10:07:28 +03:00
Pekka Enberg
344eecb7ac bindings/rust: Fix RustDoc 2025-06-26 09:58:05 +03:00
Pekka Enberg
39aa7ad1b0 cargo fmt 2025-06-26 09:48:40 +03:00
Pekka Enberg
532b9cbcdb bindings/rust: API documentation 2025-06-26 09:45:58 +03:00
Pekka Enberg
7c6fdbdfad stress: Thank you Clippy
Thank you for spotting these terrible, terrible programming mistakes.
2025-06-26 09:06:01 +03:00
Pekka Enberg
e109562449 Update COMPAT.md 2025-06-25 20:03:16 +03:00
Pekka Enberg
a45f07a6ad antithesis: Add rollback parallel test driver 2025-06-25 20:02:09 +03:00
Pekka Enberg
2123858770 bindings/python: Add Connection.rollback() 2025-06-25 20:02:09 +03:00
Pekka Enberg
5be52a5260 stress: Interactive transactions 2025-06-25 20:02:09 +03:00
Pekka Enberg
a48198ec60 Merge 'Rollback simple support' from Pere Diaz Bou
Support for simple interactive rollback like:
```sql
    create table t (x);
    insert into t values (1);
    begin;
    insert into t values (2);
    rollback;
    select * from t;
```
This PR also fixes some other issues I found while debugging:
* Checkpoint would never `clear_dirty` on pages in page cache.
* Auto commit for interactive transactions was not respected so any
`insert` after `begin` would flush frames regardless of `auto_commit`
state.
* `max_frame` on wal shared state was being updated after every
`append_frame` which was incorrect, as another transaction would be able
to use that new `max_frame` even tho the transaction could've rolled
back. Instead we update the private copy of `max_frame` and only update
it at the end.
Follow up for later are savepoints which require implementing a
subjournal to track savepoints and their modified pages.

Closes #1825
2025-06-25 20:02:09 +03:00
Pekka Enberg
0297773a02 bindings/javascript: Pass readonly flag to open 2025-06-25 18:59:44 +03:00
Pekka Enberg
4e852036c5 bindings/javascript: Drop README.md
The example is wrong (imports wrong package), but also duplicates text from
top-level README.md that will change. Let's drop this for now to avoid
confusion.
2025-06-25 18:38:35 +03:00
Forato
772ba761e6 bindings/javascript: implement readonly functionality 2025-06-25 18:36:52 +03:00
Pekka Enberg
f1197c064b Merge 'fix: use uv run instead of uvx for Pytest' from Pedro Muniz
From UV docs: https://docs.astral.sh/uv/guides/tools/#running-tools
> If you are running a tool in a
[project](https://docs.astral.sh/uv/concepts/projects/) and the tool
requires that your project is installed, e.g., when using pytest or
mypy, you'll want to use [uv
run](https://docs.astral.sh/uv/guides/projects/#running-commands)
instead of uvx. Otherwise, the tool will be run in a virtual environment
that is isolated from your project.
We were not seeing this problem in branch pull_requests, as there was a
conditional that only tested on Python 3.13.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1826
2025-06-25 17:02:51 +03:00
Pekka Enberg
66546c59a5 Merge 'core/db&pager: fix locking for initializing empty database' from Jussi Saurio
When `struct Database` is constructed, store `is_empty` as an
`Arc<AtomicBool>` - the value is true if:
1. DB size is zero
2. WAL has no frames
When `struct Pager` is constructed, this `Arc` is simply cloned. When
any connection runs a transaction it will first check `Pager::is_empty`,
and if the DB is empty, it will lock `init_lock` and then check
`is_empty` again, and if it's still true, it allocates page1 and stores
`false` in the `is_empty` `AtomicBool` and drops the lock.
---
Note that Limbo can currently have a zero DB and a WAL with frames, as
we have no special logic for folding page1 to the main DB file during
initialization.
Page 1 allocation currently happens on the first transaction (read or
write, due to having to support `select * from sqlite_schema` on an
empty DB; we should really check how SQLite actually does this.).

Closes #1830
2025-06-25 17:01:52 +03:00
Pere Diaz Bou
156027836e add another test 2025-06-25 14:01:53 +02:00
Pere Diaz Bou
c02337c8cc clear dirty pages on rollback 2025-06-25 14:01:53 +02:00
Pere Diaz Bou
34a6d236ab fix comp 2025-06-25 14:01:53 +02:00
Pere Diaz Bou
96c30be488 more clippy 2025-06-25 14:01:53 +02:00
Pere Diaz Bou
0119b0f99d clippy 2025-06-25 14:01:53 +02:00
Pere Diaz Bou
b3e0ba030d add rollback tcl tests 2025-06-25 14:01:53 +02:00
Pere Diaz Bou
a3ad138df8 checkpoint clear dirty page if it was on cache 2025-06-25 14:01:53 +02:00
Pere Diaz Bou
4c3ca2caee remove hacked cacheflush on cli app 2025-06-25 14:00:57 +02:00
Pere Diaz Bou
22f9cd695d commit_txn track rollback case 2025-06-25 14:00:57 +02:00
Pere Diaz Bou
b6a948e513 add cute log for translate input 2025-06-25 14:00:57 +02:00
Pere Diaz Bou
8517cea530 add finish append frames log 2025-06-25 14:00:57 +02:00
Pere Diaz Bou
e2df4e1d33 run cacheflush until completion cli 2025-06-25 14:00:57 +02:00
Pere Diaz Bou
bdd2010df3 autocommit rollback 2025-06-25 14:00:57 +02:00
Jussi Saurio
cb29f325e2 No need for Database::is_empty to be pub 2025-06-25 14:53:39 +03:00
Pere Diaz Bou
d66c683a4c implement rollback translation 2025-06-25 13:45:32 +02:00
Jussi Saurio
27b3ecf599 core/db&pager: fix locking for initializing empty database
When `struct Database` is constructed, store `is_empty` as an
`Arc<AtomicBool>` - the value is true if:

1. DB size is zero
2. WAL has no frames

When `struct Pager` is constructed, this `Arc` is simply cloned.
When any connection runs a transaction it will first check `is_empty`,
and if the DB is empty, it will lock `init_lock` and then check `is_empty`
again, and if it's still true, it allocates page1 and stores `false` in
the `is_empty` `AtomicBool` and drops the lock.

---

Note that Limbo can currently have a zero DB and a WAL with frames,
as we have no special logic for folding page1 to the main DB file
during initialization.

Page 1 allocation currently happens on the first transaction (read or
write, due to having to support `select * from sqlite_schema` on an
empty DB; we should really check how SQLite actually does this.).
2025-06-25 14:45:21 +03:00