Commit Graph

276 Commits

Author SHA1 Message Date
meteorgan
d2dce740f7 fix some issues about page_size 2025-04-28 16:13:07 +08:00
Jussi Saurio
9d553c50cc Merge 'allow index entry delete' from Pere Diaz Bou
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1341
2025-04-18 15:26:05 +03:00
Jussi Saurio
5628cc27a6 btree: move allocate_overflow_page to Pager impl 2025-04-14 15:25:15 +03:00
Pere Diaz Bou
651c00b4a8 allow index entry delete 2025-04-14 11:02:17 +02:00
Diego Reis
135330b736 core/pager: Fix page handling issue due change in wal type 2025-04-13 11:10:06 -03:00
Diego Reis
66e12e1c2d core/vdbe: Create OpenEphemeral bytecode
"Open a new cursor P1 to a transient table. The cursor is always opened read/write even if the main database is read-only. The ephemeral table is deleted automatically when the cursor is closed.

If the cursor P1 is already opened on an ephemeral table, the table is cleared (all content is erased)."

There is still some work to do, but this is a basic setup
2025-04-13 11:10:06 -03:00
Diego Reis
65d4c68cf2 core/pager: Wrap wal with Option 2025-04-13 11:10:05 -03:00
Diego Reis
51eb2af06a core(refactor): Add CreateBTreeFlags
Passing 1s and 0s with comments is not rustacean, and since we already follow the pattern of struct flags in other sections of the codebase it's better use it here too.
2025-04-13 01:46:30 -03:00
Pekka Enberg
53633e8b6f core/btree: Add PageContent::new() helper 2025-04-10 13:14:38 +03:00
Pekka Enberg
ed1854c8de Merge 'Request load page on insert_into_page' from Pere Diaz Bou
We assumed page was loaded because before inserting we would move there.
`NewRowId` unfortunately moves cursor to the rightmost page causing
eviction of root page -- this arose the issue with `insert_into_page`
not loading the page we were supposed to have loaded so I added
`return_if_locked_maybe_load` which is a utility macro to check if the
page is locked and if not, load it if needed.

Closes #1138
2025-04-02 18:52:25 +03:00
Pere Diaz Bou
e85fb86ff4 Request load page on insert_into_page
We assumed page was loaded because before inserting we would move there. `NewRowId` unfortunately moves cursor to the rightmost page causing eviction of root page -- this arose the issue with `insert_into_page` not loading the page we were supposed to have loaded so I added `return_if_locked_maybe_load` which is a utility macro to check if the page is locked and if not, load it if needed.
2025-04-02 16:24:53 +02:00
Pere Diaz Bou
e2d00a9f96 inline start transactions from pager and wal
Execute `SELECT 1`/limbo_execute_select_1
                        time:   [30.543 ns 30.585 ns 30.632 ns]
2025-04-02 16:18:36 +02:00
Pere Diaz Bou
141303e330 Validate cells inside a page after each operation
We need to ensure an operation doesn't transform the cells inside a page
to an invalid state. In debug mode we can enable a
`debug_validate_cells` with `#[cfg(debug_assertions)]` so that it is
skipped on release mode.

Modify pager logs
2025-04-01 11:19:23 +02:00
Daniel Boll
5fc9ccdc8c Update checkpoint result initialization and WAL frame handling
- Use `CheckpointResult::default()` instead of `CheckpointResult::new()`
- Correct WAL frame header salt and checksum handling
- Ensure frame ID is 1-based and adjust frame offset calculation
- Add `Default` implementation for `CheckpointResult`
- Use random values for WAL header salts
2025-03-25 21:38:12 -03:00
Pekka Enberg
df6af6ed79 core: Rename FileStorage to DatabaseFile 2025-03-25 11:15:16 +02:00
Pekka Enberg
52ccc36061 Merge 'Impl Copy on some types in the pager to prevent explicit clones' from Preston Thorpe
Tried to keep this as small and focused as possible, just a few that I
ran into while debugging the page cache

Closes #1107
2025-03-21 18:40:14 +02:00
Pere Diaz Bou
00ab3d1c0c Fix ordering and implement Deref 2025-03-17 10:22:42 +01:00
Pere Diaz Bou
20f5ade95e Experiment with a custom Lock for database header 2025-03-17 10:21:34 +01:00
PThorpe92
e22c86c230 Impl Copy on some types in the pager to prevent explicit clones 2025-03-12 21:57:35 -04:00
Pere Diaz Bou
825907bfac Invalidate cache entry after checkpoint of frame completes 2025-03-12 15:48:22 +01:00
Pere Diaz Bou
be3badc1f3 modify a few btree log level and add end_write_txn after checkpoint 2025-03-12 15:48:22 +01:00
Pere Diaz Bou
8daf7666d1 Make database Sync + Send 2025-03-05 14:07:48 +01:00
Pekka Enberg
f57d2b32af core: Clean up B-Tree creation code
Move page allocation to pager so that we don't need to instantiate a
cursor to create a B-Tree.
2025-03-04 18:38:06 +02:00
Pere Diaz Bou
1266994a0a balance sibling pages 2025-02-15 15:37:13 +01:00
Doug Anderson444
0aa5de6d9f rm log, switch all to tracing 2025-02-11 09:03:36 -04:00
krishvishal
ea174e32f1 chore: cargo fmt 2025-02-06 23:39:12 +05:30
krishvishal
c80007d8a0 Add local constants for free_page offsets for clarity. 2025-02-06 23:39:12 +05:30
krishvishal
8c763780a8 Added free list and free_page functionality in Pager module. Now we can add dropped pages to free list, so pages can be reused in allocate_page. 2025-02-06 23:34:12 +05:30
Pekka Enberg
e4d7474372 core: Switch to parking_lot for RwLock
We really need to make the WAL lock less expensive, but switching to
`parking_lot` is anyway something we should do.

Before:

```
Execute `SELECT 1`/Limbo
                        time:   [56.230 ns 56.463 ns 56.688 ns]
```

After:

```
Execute `SELECT 1`/Limbo
                        time:   [52.003 ns 52.132 ns 52.287 ns]
```
2025-02-04 18:38:33 +02:00
Pekka Enberg
f69804969c Merge 'Adding checkpoint result' from Sonny
### What?
adding checkpoint result returning number of pages in wal and num pages
checkpointed.
Part of #696
### Context
SQLite returns in checkpoint result of calling `pragma wal_checkpoint;`
`0|3|3` while limbo returns `0|0|0`.
https://sqlite.org/pragma.html#pragma_wal_checkpoint
- 1st col: 1 (checkpoint SQLITE_BUSY) or 0 (not busy).
- 2nd col: # modified pages written to wal file
- 3rd col: # pages moved to db after checkpoint
This PR aims to add 2nd and 3rd column to the checkpoint result.
SQLite
```
sqlite3 test.db
sqlite> pragma journal_mode=wal;
wal
sqlite> pragma journal_mode;
wal
sqlite> create table t1 (id text);
sqlite> insert into t1(id) values (1),(2);
sqlite> select * from t1;
1
2
sqlite> pragma wal_checkpoint;
0|3|3
```
Limbo
```
./target/debug/limbo test.db
Limbo v0.0.13
Enter ".help" for usage hints.
limbo> pragma journal_mode;
wal
limbo> create table t1(id text);
limbo> insert into t1(id) values (1),(2);
limbo> select * from t1;
1
2
# current the 2nd and 3rd columns are hard coded in limbo to 0
limbo> pragma wal_checkpoint;
0|0|0
```

Closes #827
2025-02-04 18:26:24 +02:00
Jussi Saurio
1f888fea4f Dont fsync the WAL on read queries 2025-02-03 20:42:50 +02:00
sonhmai
022a8d7a83 core: return checkpoint result 2025-02-03 19:02:16 +07:00
Pekka Enberg
983875c443 core: Remove database header from BTreeCursor
It's already in the pager so use it from there to reduce the size of the
`BTreeCursor` struct.
2025-01-26 16:48:12 +02:00
Jorge López
00c503dcf5 syntactic changes: lift return out of ifs 2025-01-18 19:20:11 +01:00
Jorge López
cfff4dd21c syntactic changes: fix typos in comments 2025-01-18 19:20:11 +01:00
Jorge López
86a4714711 syntactic changes: remove unneeded paths when the type is already imported 2025-01-18 18:29:12 +01:00
PThorpe92
f6cd707544 Add clippy CI, fix or ignore warnings where appropriate 2024-12-29 10:25:41 -05:00
adamnemecek
97647ff056 Clean up code to use Self
Closes #556
2024-12-29 10:07:38 +02:00
Pere Diaz Bou
a2921bd32c core: add checkpoint mode passive 2024-12-24 18:30:58 +01:00
Pere Diaz Bou
aed14117c9 core: transaction support 2024-12-24 18:04:30 +01:00
jussisaurio
b57a95752c core/btree: improve documentation 2024-12-22 23:05:25 +02:00
Pere Diaz Bou
37005a23d2 fix checkpoint 2024-12-15 18:55:23 +00:00
Pere Diaz Bou
de3449be5a add todo for cache invalidation 2024-12-15 10:03:10 +01:00
Pere Diaz Bou
39a75147d4 Page cache by page_number and frame_number
Since page cache is now shared by default, we need to cache pages by
page number and something else. I chose to go with max_frame of
connection, because this connection will have a max_frame set until from
the start of a transaction until the end of it.

With key pairs of (pgno, max_frame) we make sure each connection is
caching based on the snapshot it is at as two different connections
might have the same pageno being using but a different frame. If both
have same max_frame then they will share same page.
2024-12-13 21:57:27 +01:00
Pere Diaz Bou
1e5239a164 use correct min/max frames 2024-12-13 14:06:29 +01:00
Pere Diaz Bou
b43e8e46f6 impl sync/send for cache 2024-12-13 13:09:13 +01:00
Pere Diaz Bou
97dd95abea core: change Rc<RefCell<Page>> to Arc<Page>
This includes an inner struct in Page wrapped with Unsafe cell to access
it. This is done intentionally because concurrency control of pages is
handled by pages and not by the page itself.
2024-12-13 13:09:13 +01:00
Pere Diaz Bou
a4297702bd extract page cache to be multi threaded 2024-12-13 13:09:13 +01:00
Pere Diaz Bou
f5a1f7c800 various fixes in btree
* read_u8 now takes self.offset into account
* shift cell pointers left on balance_root with offset > 0
* fix wrong writes to page in degragment_page
2024-11-19 17:15:19 +01:00
Pere Diaz Bou
6cd0f03643 core: create databases from limbo 2024-11-15 12:09:07 +01:00