Commit Graph

778 Commits

Author SHA1 Message Date
Jussi Saurio
925a252815 Merge 'btree: Improve balance non root docs' from Jussi Saurio
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #2031
2025-07-10 15:07:18 +03:00
Jussi Saurio
0b8c5f7c91 btree/balance: extra doc context for CellArray::cell_payloads 2025-07-10 15:06:27 +03:00
Jussi Saurio
475bced4f7 btree/balance: remove obsolete todo 2025-07-10 14:58:00 +03:00
Jussi Saurio
0316b5a517 btree/balance: rename CellArray::cell_data to cell_payloads 2025-07-10 14:57:45 +03:00
Jussi Saurio
0d973d78a9 btree/balance: add a diagram about divider cell assignment and some comments 2025-07-10 14:56:59 +03:00
Pekka Enberg
7a259957ac Merge 'properly set last_checksum after recovering wal' from Pere Diaz Bou
We store `last_checksum` to do cumulative checksumming. After reading
wal for recovery, we didn't set last checksum properly in case there
were no frames so this cause us to not initialize last_checksum
properly.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2030
2025-07-10 13:54:15 +03:00
Jussi Saurio
610b743f0d btree/balance: rename CellArray::cell_count to CellArray::cell_count_up_to_page 2025-07-10 13:51:56 +03:00
Jussi Saurio
924482981c btree/balance: rename CellArray::cell_size to CellArray::cell_size_bytes 2025-07-10 13:50:14 +03:00
Jussi Saurio
68cd948056 btree/balance: add extra documentation for page update dual pass 2025-07-10 13:47:08 +03:00
Jussi Saurio
3fc51ed4d9 btree/balance: rename leaf to is_leaf 2025-07-10 13:16:14 +03:00
Jussi Saurio
d88bbd488f btree/balance: rename leaf_data to is_table_leaf 2025-07-10 13:15:29 +03:00
Pere Diaz Bou
9258d33d8b properly set last_checksum after recovering wal
We store `last_checksum` to do cumulative checksumming. After reading
wal for recovery, we didn't set last checksum properly in case there
were no frames so this cause us to not initialize last_checksum
properly.
2025-07-10 12:15:00 +02:00
Jussi Saurio
b306550a69 format 2025-07-10 13:14:57 +03:00
Jussi Saurio
5ef0127409 btree/balance: rename count_cells_in_old_pages to old_cell_count_per_page_cumulative 2025-07-10 13:14:18 +03:00
Jussi Saurio
c31ee0e628 btree/balance: rename number_of_cells_per_page to cell_count_per_page_cumulative 2025-07-10 13:12:17 +03:00
Jussi Saurio
824065a91d btree/balance: rename cells to cell_data 2025-07-10 13:10:31 +03:00
Jussi Saurio
37f2317e49 btree/balance: add comment about divider cell 2025-07-10 13:09:29 +03:00
Jussi Saurio
4d691af3ee btree/balance: clearer variable name 2025-07-10 13:08:58 +03:00
Jussi Saurio
e51f0f5466 btree/balance: improve comment 2025-07-10 13:08:35 +03:00
Jussi Saurio
201edf3668 btree/balance: add comment 2025-07-10 13:05:54 +03:00
Jussi Saurio
fd0a47dc6b btree: simplify pattern match 2025-07-10 13:05:15 +03:00
Jussi Saurio
4dc3e2100f btree: rename balance_non_root related enum variants and add docs 2025-07-10 13:02:50 +03:00
Jussi Saurio
0eeabbb748 Merge 'btree/chore: remove unnecessary parameters to .cell_get()' from Jussi Saurio
we were providing the same damn arguments to `.cell_get()` and
`.cell_get_raw_region()` over and OVER and **OVER** and `O V E R`

Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #2021
2025-07-10 12:22:37 +03:00
Jussi Saurio
c2b699c356 btree: make cell field names consistent 2025-07-09 23:43:03 +03:00
Jussi Saurio
641df7d7e9 improve my mental health by finally refactoring .cell_get() 2025-07-09 19:15:05 +03:00
meteorgan
0001348158 Minor refactoring of btree 2025-07-09 22:01:54 +08:00
Jussi Saurio
11d4489740 Merge 'sqlite3_ondisk: generalize left-child-pointer reading function to both index/table btrees' from Jussi Saurio
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #2015
2025-07-09 14:24:08 +03:00
Jussi Saurio
c752058a97 VDBE: introduce state machine for op_idx_insert for more granular IO control
Separates cursor.key_exists_in_index() into a state machine. The problem with
the main branch implementation is this:

`return_if_io!(seek)`
`return_if_io!(cursor.record())`

The latter may yield on IO and cause the seek to start over, causing an infinite
loop. With an explicit state machine we can control and prevent this.
2025-07-09 11:43:18 +03:00
Jussi Saurio
c13b2d5d90 sqlite3_ondisk: generalize left-child-pointer reading function to both index/table btrees 2025-07-09 11:07:42 +03:00
Pere Diaz Bou
f7465f665d add checkpoint lock to wal 2025-07-08 17:53:04 +02:00
meteorgan
99e0cf0603 add a constant MINIMUM_CELL_SIZE 2025-07-08 22:57:20 +08:00
meteorgan
04575456a9 fix Minimum cell size must not be less than 4 2025-07-08 22:57:20 +08:00
Jussi Saurio
3ab5f07389 btree: fix incorrect comparison implementation in key_exists_in_index()
1. current implementation did not use the custom PartialOrd implementation
   for RefValue
2. current implementation did not take collation into account
2025-07-08 11:58:57 +03:00
Pekka Enberg
1907df825c Merge 'Use binary search in find_cell()' from Ihor Andrianov
Find cell using  bin search

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1875
2025-07-08 10:22:26 +03:00
pedrocarlo
367002fb72 rename change_schema to schema_did_change 2025-07-07 11:58:16 -03:00
pedrocarlo
d8ad4a27f8 only finish appending frames when we are done in cacheflush 2025-07-07 11:53:45 -03:00
pedrocarlo
b85687658d change instrumentation level to INFO 2025-07-07 11:53:45 -03:00
pedrocarlo
4639a4565f change max_frame count only after wal sync in cacheflush 2025-07-07 11:53:45 -03:00
pedrocarlo
46f59e4f0f add more instrumentation + add faults to shrunk plan 2025-07-07 11:53:45 -03:00
pedrocarlo
7c10ac01e6 do_allocate_page should return a Result 2025-07-07 11:53:45 -03:00
pedrocarlo
711b1ef114 make all run_once be run under statement or connection so that rollback is called 2025-07-07 11:51:25 -03:00
pedrocarlo
5559c45011 more instrumentation + write counter should decrement if pwrite fails 2025-07-07 11:50:21 -03:00
pedrocarlo
9632ab0a41 rollback transaction when we fail in step 2025-07-07 11:50:21 -03:00
pedrocarlo
897426a662 add error tracing to relevant functions + rollback transaction in step_end_write_txn + make move_to_root return result 2025-07-07 11:50:21 -03:00
pedrocarlo
d82b526a5c fix infinite loop with write counter 2025-07-07 11:50:21 -03:00
pedrocarlo
db005c81a0 add option to disable wal checkpoint 2025-07-03 12:04:17 -03:00
Ihor Andrianov
650c85ccd7 save binary search state for reentrant execution 2025-07-03 15:08:16 +03:00
Pekka Enberg
90e035b6b0 Merge 'Rollback schema support' from Pere Diaz Bou
Fixes #1890
Once rollback was implement we quickly saw that it lacked support for
schema changes so we had to re-estructure things a bit.
## Example of failure:
```bash
turso> begin;
turso> create table t(x);
turso> rollback;
turso> pragma integrity_check;
thread 'main' panicked at core/storage/sqlite3_ondisk.rs:386:36:
called `Result::unwrap()` on an `Err` value: Corrupt("Invalid page type: 83")
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
This happened because it thought table `t` existed because we didn't
rollback that schema.
## Changes:
* The most important change: now every connection has a private copy of
schema. On write txn commit we update a global schema shared between
connections in order for new connections to get updated version from
there. In case of rollback, we simply change connection's schema to
previous version. This change allowed us to remove locks for schema
private copy and keeping schema changes locally in case of concurrency.
 Sqlite does things differently, they lazily parse schema in case of
outdated schema, this many schema changes to trigger reading schema from
db file which is slow. If we are able to keep local copy in memory, even
when if we add multiprocessing, it will speed up schema reloading by a
bunch.
* `schema_cookie` is now update for every schema change
* `Insn::ParseSchema` had a nasty bug where it would commit all the
changes made in a query that changed a schema, we fixed that by setting
`auto_commit` to `false` before parsing schema, and setting it back to
previous value once schema is parsed.

Closes #1928
2025-07-03 14:18:00 +03:00
Pere Diaz Bou
5d856499c4 move update schema global on commit and not on rollback txn 2025-07-03 12:36:48 +02:00
Pere Diaz Bou
c799396c3d rollback schema in connection 2025-07-03 12:36:48 +02:00