Commit Graph

297 Commits

Author SHA1 Message Date
Pekka Enberg
f69804969c Merge 'Adding checkpoint result' from Sonny
### What?
adding checkpoint result returning number of pages in wal and num pages
checkpointed.
Part of #696
### Context
SQLite returns in checkpoint result of calling `pragma wal_checkpoint;`
`0|3|3` while limbo returns `0|0|0`.
https://sqlite.org/pragma.html#pragma_wal_checkpoint
- 1st col: 1 (checkpoint SQLITE_BUSY) or 0 (not busy).
- 2nd col: # modified pages written to wal file
- 3rd col: # pages moved to db after checkpoint
This PR aims to add 2nd and 3rd column to the checkpoint result.
SQLite
```
sqlite3 test.db
sqlite> pragma journal_mode=wal;
wal
sqlite> pragma journal_mode;
wal
sqlite> create table t1 (id text);
sqlite> insert into t1(id) values (1),(2);
sqlite> select * from t1;
1
2
sqlite> pragma wal_checkpoint;
0|3|3
```
Limbo
```
./target/debug/limbo test.db
Limbo v0.0.13
Enter ".help" for usage hints.
limbo> pragma journal_mode;
wal
limbo> create table t1(id text);
limbo> insert into t1(id) values (1),(2);
limbo> select * from t1;
1
2
# current the 2nd and 3rd columns are hard coded in limbo to 0
limbo> pragma wal_checkpoint;
0|0|0
```

Closes #827
2025-02-04 18:26:24 +02:00
Jussi Saurio
1f888fea4f Dont fsync the WAL on read queries 2025-02-03 20:42:50 +02:00
sonhmai
022a8d7a83 core: return checkpoint result 2025-02-03 19:02:16 +07:00
Pekka Enberg
983875c443 core: Remove database header from BTreeCursor
It's already in the pager so use it from there to reduce the size of the
`BTreeCursor` struct.
2025-01-26 16:48:12 +02:00
Jorge López
00c503dcf5 syntactic changes: lift return out of ifs 2025-01-18 19:20:11 +01:00
Jorge López
cfff4dd21c syntactic changes: fix typos in comments 2025-01-18 19:20:11 +01:00
Jorge López
86a4714711 syntactic changes: remove unneeded paths when the type is already imported 2025-01-18 18:29:12 +01:00
PThorpe92
f6cd707544 Add clippy CI, fix or ignore warnings where appropriate 2024-12-29 10:25:41 -05:00
adamnemecek
97647ff056 Clean up code to use Self
Closes #556
2024-12-29 10:07:38 +02:00
Pere Diaz Bou
a2921bd32c core: add checkpoint mode passive 2024-12-24 18:30:58 +01:00
Pere Diaz Bou
aed14117c9 core: transaction support 2024-12-24 18:04:30 +01:00
jussisaurio
b57a95752c core/btree: improve documentation 2024-12-22 23:05:25 +02:00
Pere Diaz Bou
37005a23d2 fix checkpoint 2024-12-15 18:55:23 +00:00
Pere Diaz Bou
de3449be5a add todo for cache invalidation 2024-12-15 10:03:10 +01:00
Pere Diaz Bou
39a75147d4 Page cache by page_number and frame_number
Since page cache is now shared by default, we need to cache pages by
page number and something else. I chose to go with max_frame of
connection, because this connection will have a max_frame set until from
the start of a transaction until the end of it.

With key pairs of (pgno, max_frame) we make sure each connection is
caching based on the snapshot it is at as two different connections
might have the same pageno being using but a different frame. If both
have same max_frame then they will share same page.
2024-12-13 21:57:27 +01:00
Pere Diaz Bou
1e5239a164 use correct min/max frames 2024-12-13 14:06:29 +01:00
Pere Diaz Bou
b43e8e46f6 impl sync/send for cache 2024-12-13 13:09:13 +01:00
Pere Diaz Bou
97dd95abea core: change Rc<RefCell<Page>> to Arc<Page>
This includes an inner struct in Page wrapped with Unsafe cell to access
it. This is done intentionally because concurrency control of pages is
handled by pages and not by the page itself.
2024-12-13 13:09:13 +01:00
Pere Diaz Bou
a4297702bd extract page cache to be multi threaded 2024-12-13 13:09:13 +01:00
Pere Diaz Bou
f5a1f7c800 various fixes in btree
* read_u8 now takes self.offset into account
* shift cell pointers left on balance_root with offset > 0
* fix wrong writes to page in degragment_page
2024-11-19 17:15:19 +01:00
Pere Diaz Bou
6cd0f03643 core: create databases from limbo 2024-11-15 12:09:07 +01:00
Pere Diaz Bou
c46bd63b5a core: drop mutex on contents
There is no need to have mutexes on buffers, we will introduce mutexes
if we want later on a file level to introduce serializability.
2024-11-13 19:13:15 +01:00
Pere Diaz Bou
94a45eab9e checkpoint inflight 2024-11-13 12:04:54 +00:00
Pere Diaz Bou
87957bf4f2 checkpoint log state 2024-11-13 11:24:07 +00:00
Pere Diaz Bou
48f0e72e14 checkpoint on drop connection 2024-11-12 17:03:30 +01:00
Pere Diaz Bou
807496a146 core: load pages if not loaded
Since pages are now tracked with a single centralized structure, it is
possible for a page to have been unloaded a be kept in memory for
metadata purposes like going back in the stack.

Using the example of going to a parent page which was unloaded for
whatever reason: in this case we need to check if it's loaded, if not,
we load it. Locked still means the same thing, loaded just means the
contents of a page are present in memory and if they are present, they
must be in cache.
2024-11-12 14:44:25 +01:00
Pere Diaz Bou
cd2b61d838 btree: cursor with lineal stack structure
Removed MemPage from the code in favor of an array to encode the stack
of the cursor. This is both simpler and better in terms of memory
access.
2024-11-12 10:21:22 +01:00
Pere Diaz Bou
eb8c462c5f fix io submission on cacheflush 2024-11-06 16:25:42 +00:00
Pere Diaz Bou
8eb3c89227 wasm,sim fixes 2024-11-05 15:41:30 +01:00
Pere Diaz Bou
a85d599c65 state machine cacheflush 2024-11-05 15:29:54 +01:00
Pere Diaz Bou
70a4ccd8bb fix linux completion match 2024-11-05 15:29:54 +01:00
Pere Diaz Bou
fc65c5096d cacheflush state machine 2024-11-05 15:29:54 +01:00
Pere Diaz Bou
f009eb35c6 suspendable checkpoint 2024-11-05 15:29:54 +01:00
Pere Diaz Bou
c0e51c4ca6 wip wal 2024-11-05 15:29:53 +01:00
김선우
28884181be Fix clippy 2024-09-15 16:23:27 +09:00
jussisaurio
b6e88ca883 cargo clippy --fix --allow-dirty && cargo fmt 2024-09-15 09:35:39 +03:00
Pere Diaz Bou
e9bc4b04a7 overflow pages support 2024-09-13 20:32:33 +02:00
Pere Diaz Bou
d87f9c9774 core: multiple level btree page split
Signed-off-by: Pere Diaz Bou <pere-altea@hotmail.com>
2024-09-05 20:50:30 +02:00
Pekka Enberg
89079d1ccd Merge 'core: fix clippy' from Sonny
Closes #268
2024-08-03 17:53:51 +03:00
sonhmai
0e7bd95e4e core: fix clippy 2024-08-03 20:14:26 +07:00
Pekka Enberg
ed4116e7c2 core: Introduce Wal trait
We're going to need it for WebAssembly anyway, which does not have
standard filesystem support.
2024-08-03 12:34:10 +03:00
Pekka Enberg
090a577dd5 core: Move DatabaseStorage to storage/database.rs 2024-08-03 10:41:10 +03:00
Pekka Enberg
8a54e31803 core: Rename PageIO to DatabaseStorage 2024-08-03 10:33:52 +03:00
Pekka Enberg
4349b946e5 core: Eliminate PageSource wrapper
The PageSource wrapper is useless. Let's inline it and use PageIO
directly.
2024-08-03 10:27:20 +03:00
Pekka Enberg
0bf12ec1b3 core: Move buffer_pool.rs to storage module 2024-08-01 11:53:14 +03:00
Pekka Enberg
ed1c23bfe6 core: Move wal.rs to storage module 2024-08-01 11:52:50 +03:00
Pekka Enberg
f8a43361db core: Move pager.rs to storage module 2024-08-01 11:52:50 +03:00