Closes: #1415
### What this PR does
1. Removes database initialization from the `read_tx` function.
2. Adds checks for database initialization when executing `.schema`,
`.indexes`, `.tables` and `.import` commands, as they rely on
`sqlite_schema` table.
### About the second issue
I think we have another solution for the second issue: create the
`sqlite_schema` table in `Schema` only during page1 initialization,
rather than during `Schema` initialization.
#### Pros
This approach has the advantage of unifying the logic for the
`sqlite_schema` table with other user tables when running `select`
statements
#### Cons
- we still need to check error codes for commands like `.schema`.
- this approach may increase the complexity of the `pager`
implementation.
I'd like to hear your thoughts and feedback.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#2099
Closes#2227 , enables fixing #2225
## What
Although we cleared overflow pages on DELETE, we never did it for
INSERT/UPDATE, which means any overflow pages were left dangling and not
added to freelist.
## Why is this a problem
This means that we are not able to reuse these pages to solve #2225,
causing massive bloat in the DB when UPDATEs are executed.
## Fix
Clear overflow pages when `BTreeCursor::insert()` overwrites a cell.
Needed a new state machine for `overwrite_cell` + new `WriteState`
variants
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#2230
Closes#2241
## What
When an index interior cell is deleted, it steals the leaf cell with the
largest key in its left subtree, deletes the old interior cell and then
replaces it with the stolen cell. This ensures the binary-search-tree
aspect of the btree remains correct. However, this can cause a situation
where both are true:
1. The leaf page is now UNDERFULL and must be rebalanced
2. The leaf's IMMEDIATE parent page is now OVERFULL and must be
rebalanced
## Why is this a problem
We simply didn't support the case where:
- Leaf page P is unbalanced and rebalancing starts on it
- Its immediate parent is ALSO unbalanced and _overflows_.
We had an assertion against this happening (see #2241)
## The fix
Allow exactly 1 overflow cell in the parent under very particular
conditions:
1. The parent page must be an index interior page
2. The parent must be positioned exactly at the divider cell whose left
child page underflows
This is the _only_ case where the immediate parent of a page about to
undergo rebalancing can have overflow cells.
## Implementation details
The parent overflow cell is folded into `cell_array` fairly early on and
`parent.overflow_cells` is cleared. However we need to be careful with
`cell_idx` for dividers other than the overflow cell because they get
shifted left on the page in `drop_cell()`. I've added a long comment
about this.
## Testing
Adds fuzz test that does inserts and deletes on an index btree and
asserts that all the expected keys are found at the end in the right
order. This test runs into this case quite frequently so I was able to
verify it.
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#2243
Let's make sure we don't end up in a weird situation by appending frames
one by one and we can later think of optimizations.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#2034
SQLite behavior is: if another connection has modified the DB when a
read tx starts, it must clear its page cache due to the potentiality
of there being stale versions of pages in it.
In the future, we may want to do either:
1. a more granular invalidation logic for per-conn cache, or
2. a shared versioned page cache
But right now we must follow SQLite to make our current behavior not
corrupt data
This PR implements missing raw WAL API from LibSQL for future use for
offline-sync feature:
1. `wal_insert_begin` - begin WAL session by opening WAL read/write
transaction
2. `wal_insert_end` - finish WAL session by closing WAL transaction
opened by `wal_insert_begin` call
3. `wal_insert_frame` - insert frame `frame_no` with raw content `frame`
(WAL frame included)
For now any schema changes will not be reflected after
`wal_insert_frame` because `turso-db` do not re-parse schema without
need. I will fix this in follow up PR.
Reviewed-by: Pekka Enberg <penberg@iki.fi>
Closes#2231
This PR fixes `wal_read_frame_raw` API
Before, implementation of raw read API read only page content - which is
not enough as we also need page_no and size_after fields from the
header. This PR fixes that and also make few adjustments in the
signatures.
Reviewed-by: Preston Thorpe (@PThorpe92)
Closes#2229
- When an interior index cell is replaced, it can cause the page where
the
replacement happens to overflow OR underflow. On `main` we did not check
this case, because
the interior cell replacement always moves the cursor to a leaf, and if
the leaf
doesn't underflow, then no further balancing happens.
- The solution is to ALWAYS check whether the interior page where the
replacement
happens is underflowing OR overflowing, and balance that page regardless
of whether
the leaf page where the replacement was taken underflows or not.
So summary:
- InteriorCellReplacement: cell deleted from Interior page I,
replacement cell taken from Leaf L
and inserted back to Interior page I.
- If Leaf L underflows:
* balance it first
* then balance I if it overflows OR underflows
- If Leaf L does NOT underflow:
* balance I anyway if it overflows OR underflows
Closes https://github.com/tursodatabase/turso/issues/1701
Closes https://github.com/tursodatabase/turso/issues/2167
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#2168
