Commit Graph

95 Commits

Author SHA1 Message Date
PThorpe92
e22c86c230 Impl Copy on some types in the pager to prevent explicit clones 2025-03-12 21:57:35 -04:00
Pere Diaz Bou
f9916e8149 update max frame in case we got a read lock with outdated max frame 2025-03-12 15:48:22 +01:00
Pere Diaz Bou
7aae885af4 add logs to wal locks 2025-03-05 22:32:59 +01:00
Pere Diaz Bou
8daf7666d1 Make database Sync + Send 2025-03-05 14:07:48 +01:00
Doug Anderson444
0aa5de6d9f rm log, switch all to tracing 2025-02-11 09:03:36 -04:00
PThorpe92
75898027a0 Remove unnecessary reference counting from completion io callbacks 2025-02-08 08:20:19 -05:00
Pekka Enberg
e4d7474372 core: Switch to parking_lot for RwLock
We really need to make the WAL lock less expensive, but switching to
`parking_lot` is anyway something we should do.

Before:

```
Execute `SELECT 1`/Limbo
                        time:   [56.230 ns 56.463 ns 56.688 ns]
```

After:

```
Execute `SELECT 1`/Limbo
                        time:   [52.003 ns 52.132 ns 52.287 ns]
```
2025-02-04 18:38:33 +02:00
sonhmai
022a8d7a83 core: return checkpoint result 2025-02-03 19:02:16 +07:00
sonhmai
1c803aa079 feat: add wal checkpoint result 2025-02-03 18:57:22 +07:00
sonhmai
fcd893284b chore: fix typos 2025-01-23 11:25:01 +07:00
sonhmai
cb631dafdc feat: wire checkpoint to bytecode execution 2025-01-20 08:34:13 +07:00
Jorge López
cfff4dd21c syntactic changes: fix typos in comments 2025-01-18 19:20:11 +01:00
Jorge López
86a4714711 syntactic changes: remove unneeded paths when the type is already imported 2025-01-18 18:29:12 +01:00
PThorpe92
f6cd707544 Add clippy CI, fix or ignore warnings where appropriate 2024-12-29 10:25:41 -05:00
Pekka Enberg
244326ee57 core: Remove unused imports 2024-12-27 10:55:31 +02:00
Pere Diaz Bou
a2921bd32c core: add checkpoint mode passive 2024-12-24 18:30:58 +01:00
Pere Diaz Bou
3bce282352 respect max_frame on checkpoint 2024-12-24 18:18:17 +01:00
Pere Diaz Bou
aed14117c9 core: transaction support 2024-12-24 18:04:30 +01:00
Pere Diaz Bou
a4d7fee090 typos 2024-12-15 22:54:32 +01:00
Pere Diaz Bou
78d8bb1fa6 comments for everyone 2024-12-15 22:52:49 +01:00
Pere Diaz Bou
37005a23d2 fix checkpoint 2024-12-15 18:55:23 +00:00
Pere Diaz Bou
c6481c4250 add missing propagation error 2024-12-15 10:02:08 +01:00
Pere Diaz Bou
e75b90e35b add contents to checkpoint page 2024-12-15 10:01:22 +01:00
Pere Diaz Bou
39a75147d4 Page cache by page_number and frame_number
Since page cache is now shared by default, we need to cache pages by
page number and something else. I chose to go with max_frame of
connection, because this connection will have a max_frame set until from
the start of a transaction until the end of it.

With key pairs of (pgno, max_frame) we make sure each connection is
caching based on the snapshot it is at as two different connections
might have the same pageno being using but a different frame. If both
have same max_frame then they will share same page.
2024-12-13 21:57:27 +01:00
Pere Diaz Bou
1e5239a164 use correct min/max frames 2024-12-13 14:06:29 +01:00
Pere Diaz Bou
97dd95abea core: change Rc<RefCell<Page>> to Arc<Page>
This includes an inner struct in Page wrapped with Unsafe cell to access
it. This is done intentionally because concurrency control of pages is
handled by pages and not by the page itself.
2024-12-13 13:09:13 +01:00
Pere Diaz Bou
c816186326 fmt 2024-12-13 13:09:13 +01:00
Pere Diaz Bou
3fda2d09b9 Extract multi threaded part from WalFile to WalFileShared
Since we expect to ensure thread safety between multiple threads in the
future, we extract what is important to be shared between multiple
connections with regards to WAL.

This is WIP so I just put whatever feels like important behind a RwLock
but expect this to change to Atomics in the future as needed. Maybe even
these locks might disappear because they will be better served with
transaction locks.
2024-12-13 13:09:13 +01:00
Pere Diaz Bou
a4297702bd extract page cache to be multi threaded 2024-12-13 13:09:13 +01:00
Lauri Virtanen
a7100d8e9b Autofix clippy issues with cargo fix --clippy 2024-11-24 20:24:47 +02:00
Pere Diaz Bou
05a05cfc63 endian naming changes 2024-11-21 11:55:36 +01:00
Pere Diaz Bou
6c14c50d41 wal: checksums
Implemeted checksums so that sqlite3 is able to read our WAL. This also
helps with future work on proper recovery of WAL.

Create some frames with CREATE TABLE and kill the process so that there
is no checkpoint.
```
Limbo v0.0.6
Enter ".help" for usage hints.
limbo> create table x(x);
limbo> [1]    15910 killed     cargo run xlimbo.db
```

Now sqlite3 is able to recover from this WAL created in limbo:

```
sqlite3 xlimbo.db
SQLite version 3.43.2 2023-10-10 13:08:14
Enter ".help" for usage hints.
sqlite> .schema
CREATE TABLE x (x);
```
2024-11-20 20:12:32 +01:00
Pere Diaz Bou
87c80b2a07 use core wal file for wasm 2024-11-14 17:54:20 +01:00
Pere Diaz Bou
e2276c2e9d O_DIRECT disable on WAL 2024-11-13 13:45:42 +00:00
Pere Diaz Bou
48f0e72e14 checkpoint on drop connection 2024-11-12 17:03:30 +01:00
Pere Diaz Bou
807496a146 core: load pages if not loaded
Since pages are now tracked with a single centralized structure, it is
possible for a page to have been unloaded a be kept in memory for
metadata purposes like going back in the stack.

Using the example of going to a parent page which was unloaded for
whatever reason: in this case we need to check if it's loaded, if not,
we load it. Locked still means the same thing, loaded just means the
contents of a page are present in memory and if they are present, they
must be in cache.
2024-11-12 14:44:25 +01:00
Pere Diaz Bou
0b46648c0e write wal header on init 2024-11-07 13:29:57 +01:00
Pere Diaz Bou
8eb3c89227 wasm,sim fixes 2024-11-05 15:41:30 +01:00
Pere Diaz Bou
a85d599c65 state machine cacheflush 2024-11-05 15:29:54 +01:00
Pere Diaz Bou
fc65c5096d cacheflush state machine 2024-11-05 15:29:54 +01:00
Pere Diaz Bou
f009eb35c6 suspendable checkpoint 2024-11-05 15:29:54 +01:00
Pere Diaz Bou
c0e51c4ca6 wip wal 2024-11-05 15:29:53 +01:00
Pekka Enberg
a290b2f102 core: Open WAL file and parse header 2024-08-03 12:48:16 +03:00
Pekka Enberg
ed4116e7c2 core: Introduce Wal trait
We're going to need it for WebAssembly anyway, which does not have
standard filesystem support.
2024-08-03 12:34:10 +03:00
Pekka Enberg
ed1c23bfe6 core: Move wal.rs to storage module 2024-08-01 11:52:50 +03:00