Commit Graph

507 Commits

Author SHA1 Message Date
Pere Diaz Bou
82e5597b00 long fuzz tests ci on btree changes
The idea is simple, if you modify the btree, we should verify fuzz tests
with long number of iterations to decrease the chance of a regression
2025-05-16 11:26:00 +02:00
Pekka Enberg
a6270e8a6c Merge 'Add libsql_wal_frame_count() API' from Pekka Enberg
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1489
2025-05-15 12:01:45 +03:00
Pekka Enberg
2127de3422 cargo fmt 2025-05-15 12:00:55 +03:00
Pekka Enberg
524a523036 sqlite3: Add libsql_wal_frame_count() API 2025-05-15 11:43:44 +03:00
Pekka Enberg
9303262dfd OwnedValue -> Value 2025-05-15 11:43:23 +03:00
Pere Diaz Bou
ef65275bda Merge 'test page_free_array' from Pere Diaz Bou
Simply add a fuzz test to test free_array works as intended

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1480
2025-05-15 10:19:09 +02:00
Pekka Enberg
e3f71259d8 Rename OwnedValue -> Value
We have not had enough merge conflicts for a while so let's do a
tree-wide rename.
2025-05-15 09:59:46 +03:00
pedrocarlo
9fc9415b20 use Jussi's code to avoid cloning immutable record 2025-05-14 13:30:39 -03:00
pedrocarlo
72cc0fcdcb fixes and comments 2025-05-14 13:30:39 -03:00
pedrocarlo
b2615d7739 add CursorValidState and only save context in delete when rebalancing 2025-05-14 13:30:39 -03:00
pedrocarlo
814508981c fixing more rebase issues and cleaning up code. Save cursor context when calling delete for later use when needed 2025-05-14 13:30:39 -03:00
pedrocarlo
05f4ca28cc btree rewind and next fix. Keep track of rowids seen to avoid infinite loop 2025-05-14 13:30:39 -03:00
pedrocarlo
c146877344 add sqlite debug cli for nix. Fix cursor delete panic. Add tracing for cell indices in btree 2025-05-14 13:30:39 -03:00
Pere Diaz Bou
5025655f75 debug assertions 2025-05-14 13:53:51 +02:00
Pere Diaz Bou
c3432e1cc3 fmt 2025-05-14 13:23:12 +02:00
Pere Diaz Bou
13a120530e comment shift_cells_left
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-05-14 13:20:11 +02:00
Pere Diaz Bou
0d8a94d49d expect parse seed in rng_from_time_or_env
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-05-14 13:19:20 +02:00
Pere Diaz Bou
34c74e34d9 test page_free_array
Simply add a fuzz test to test free_array works as intended
2025-05-14 12:48:59 +02:00
Jussi Saurio
2b6b09d435 Merge 'btree: Coalesce free blocks in page_free_array()' from Mohamed Hossam
Coalesce adjacent free blocks during `page_free_array()` in
`core/storage/btree`.
Instead of immediately passing free cells to `free_cell_range()`, buffer
up to 10 free cells and try to merge adjacent free blocks. Break on the
first merge to avoid time complexity, `free_cell_range()` coalesces
blocks afterwards anyways. This follows SQLite's [`pageFreeArray()`](htt
ps://github.com/sqlite/sqlite/blob/d7324103b196c572a98724a5658970b4000b8
c39/src/btree.c#L7729) implementation.
Removed this TODO:
```rust
fn page_free_array( . . )
    .
    .
    // TODO: implement fancy smart free block coalescing procedure instead of dumb free to
    // then defragment
```

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1448
2025-05-12 16:52:11 +03:00
m0hossam
70b9438f80 Add comments 2025-05-12 15:58:36 +03:00
pedrocarlo
977d09fd36 small fixes 2025-05-10 22:23:01 -03:00
pedrocarlo
342bf51c88 remove state machine for count 2025-05-10 22:23:01 -03:00
pedrocarlo
655ceeca45 correct count implementation 2025-05-10 22:23:01 -03:00
m0hossam
aefce20460 Try to coalesce free blocks before freeing 2025-05-04 22:14:51 +03:00
PThorpe92
d4cf8367ba Wrap return_if_locked in balance non root in debug assertion cfg 2025-05-02 10:55:00 -04:00
PThorpe92
f025f7e91e Fix panic on async io due to reading locked page 2025-05-02 10:55:00 -04:00
Jussi Saurio
6096cfb3d8 Merge 'Add PRAGMA schema_version' from Anton Harniakou
This PR adds `PRAGMA schema_version` to get the value of the schema-
version integer at offset 40 in the database header.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1427
2025-05-01 10:50:02 +03:00
Jussi Saurio
a525feb7ad Merge 'Fix: allow page_size=65536' from meteorgan
Since `page_size` in `DatabaseHeader` can be 1 representing 65526 bytes,
it can't be used it directly.  Additionally, we should use `u32` instead
of `u16` or `usize` in other contexts.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1411
2025-05-01 10:46:19 +03:00
Anton Harniakou
525b7fdbaa Add PRAGMA schema_version 2025-04-30 09:41:04 +03:00
Pere Diaz Bou
a30241ca91 Add state machine for op_idx_delete + DeleteState simplification
DeleteState had a bit too many unnecessary states so I removed them.
Usually we care about having a different state when I/O is triggered
requiring a state to be stored for later.

Furthermore, there was a bug with op_idx_delete where if balance is
triggered, op_idx_delete wouldn't be re-entrant. So a state machine was
added to prevent that from happening.
2025-04-29 14:58:20 +03:00
meteorgan
d1a50f8a69 skip unneccessary conversion 2025-04-28 16:13:07 +08:00
meteorgan
d2dce740f7 fix some issues about page_size 2025-04-28 16:13:07 +08:00
Jussi Saurio
3459c1f7dd Merge 'btree/tablebtree_move_to: micro-optimizations' from Jussi Saurio
```bash
jussi@Jussis-MacBook-Pro limbo % git co main && cargo build --bin limbo --release && hyperfine --shell=none --warmup 5 './target/release/limbo TPC-H.db "select l_orderkey, 3 as revenue, o_orderdate, o_shippriority from lineitem, orders, customer where c_mktsegment = '\''FURNITURE'\'' and c_custkey = o_custkey and l_orderkey = o_orderkey and o_orderdate < cast('\''1995-03-29'\'' as datetime) and l_shipdate > cast('\''1995-03-29'\'' as datetime);"'

...

Benchmark 1: ./target/release/limbo TPC-H.db "select l_orderkey, 3 as revenue, o_orderdate, o_shippriority from lineitem, orders, customer where c_mktsegment = 'FURNITURE' and c_custkey = o_custkey and l_orderkey = o_orderkey and o_orderdate < cast('1995-03-29' as datetime) and l_shipdate > cast('1995-03-29' as datetime);"
  Time (mean ± σ):      2.104 s ±  0.006 s    [User: 1.952 s, System: 0.151 s]
  Range (min … max):    2.094 s …  2.115 s    10 runs

jussi@Jussis-MacBook-Pro limbo % git co move-to-micro-opt && cargo build --bin limbo --release && hyperfine --shell=none --warmup 5 './target/release/limbo TPC-H.db "select l_orderkey, 3 as revenue, o_orderdate, o_shippriority from lineitem, orders, customer where c_mktsegment = '\''FURNITURE'\'' and c_custkey = o_custkey and l_orderkey = o_orderkey and o_orderdate < cast('\''1995-03-29'\'' as datetime) and l_shipdate > cast('\''1995-03-29'\'' as datetime);"'

...

Benchmark 1: ./target/release/limbo TPC-H.db "select l_orderkey, 3 as revenue, o_orderdate, o_shippriority from lineitem, orders, customer where c_mktsegment = 'FURNITURE' and c_custkey = o_custkey and l_orderkey = o_orderkey and o_orderdate < cast('1995-03-29' as datetime) and l_shipdate > cast('1995-03-29' as datetime);"
  Time (mean ± σ):      1.883 s ±  0.012 s    [User: 1.733 s, System: 0.146 s]
  Range (min … max):    1.866 s …  1.908 s    10 runs
```

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1408
2025-04-28 10:27:59 +03:00
Pere Diaz Bou
63a94e7c62 Merge 'Emit IdxDelete instruction and some fixes on seek after deletion' from Pere Diaz Bou
Previously `DELETE FROM ...` only emitted deletes for main table, but
this is incorrect as we want to remove entries from index tables as
well.

Closes #1383
2025-04-28 09:13:54 +03:00
Jussi Saurio
46d45a6bf4 don't recompute cell_count 2025-04-26 14:50:42 +03:00
Jussi Saurio
75c6678a06 sqlite3_ondisk: use debug asserts for cell_table_interior_read... funcs 2025-04-26 14:50:42 +03:00
Jussi Saurio
ac1bc17ea4 btree/tablebtree_seek: remove some more useless calls to set_cell_index() 2025-04-26 13:41:30 +03:00
Jussi Saurio
5060f1a1fa don't use integer division in binary search halfpoint calculation 2025-04-26 12:38:15 +03:00
Jussi Saurio
23ae5a27c4 btree/tablebtree_move_to: micro-optimizations 2025-04-26 11:32:24 +03:00
Jussi Saurio
3553d05c32 Merge 'Give name to hard-coded page_size values' from Anton Harniakou
Related to #1379
I guess there are more hard-coded values.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1404
2025-04-25 22:03:43 +03:00
Anton Harniakou
dd7c0ad1c8 Give name to hard-coded page_size values 2025-04-25 14:15:15 +03:00
Jussi Saurio
7921d7c2e0 types: refactor serialtype again to make it faster 2025-04-24 17:28:31 +03:00
Jussi Saurio
04adf8242a faster validate 2025-04-24 16:05:12 +03:00
Jussi Saurio
af6a783f4d core/types: remove duplicate serialtype implementation 2025-04-24 15:38:47 +03:00
Jussi Saurio
dc3e97887f Merge 'replace vec with array in btree balancing' from Lâm Hoàng Phúc
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1354
2025-04-24 11:22:07 +03:00
Jussi Saurio
af703110f8 btree: remove extra iter_dir argument that can be derived from seek_op 2025-04-23 17:38:48 +03:00
Jussi Saurio
044339efc7 btree: rename tablebtree_move_to_binsearch -> tablebtree_move_to 2025-04-23 17:35:22 +03:00
Jussi Saurio
8c338438dd btree: use binary search for index interior cell seek 2025-04-23 17:34:32 +03:00
Jussi Saurio
7a133f422f btree: use binary search for index leaves 2025-04-23 17:34:32 +03:00
Jussi Saurio
8743dcd0da btree: extract indexbtree_seek() into a function like tablebtree_seek() 2025-04-23 17:34:32 +03:00