This PR adds support for `INSERT` queries with explicit value for
`rowid` column (not thought rowid alias):
```
turso> create table t(x, y, z);
turso> insert into t(rowid, x, y, z) values (10, 1, 2, 3);
turso> select rowid, * from t;
┌───────┬───┬───┬───┐
│ rowid │ x │ y │ z │
├───────┼───┼───┼───┤
│ 10 │ 1 │ 2 │ 3 │
└───────┴───┴───┴───┘
```
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#2239
Before this update, the entire immutable record was **fully**
deserialized **every** time it was compared in the sorter.
This PR extends the sorter with incremental deserialization of record
keys, only when needed and only if they weren’t already deserialized in
a previous iteration.
I hate that we panic on failed deserialization in `cmp`, but
unfortunately, I can’t return `Result` as part of this interface.
Looking for feedback around a better way to handle this.
Alternatively, I could store the deserialization error as part of
`SortableImmutableRecord` and check it before returning the record in
`next`, thereby deferring the error handling. The downside of this
approach is that it complicates debugging, since the error will be
completely decoupled from the place where it occurs.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#2207
# Fix SUM aggregate function for mixed types
Fixes#2133
The SUM aggregate function was returning incorrect results when
processing tables with mixed numeric and non-numeric values. According
to SQLite documentation:
> "If any input to sum() is neither an integer nor a NULL, then sum()
returns a floating point value"
[*](https://sqlite.org/lang_aggfunc.html)
Now both SQLite and Turso yield the same output of 44.0.
--
I modified `Sum` to increment only for numeric values, skipping non-
numeric values. However, if we have mixed numeric values or non-numeric
values, we return a float output. Added a flag to keep track of it.
as pointed out by @FHaggs , If there are no non-NULL input rows then
sum() returns NULL but total() returns 0.0. I decided to include it in
this PR as well. Empty was such a natural test case.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#2182
This PR implements missing raw WAL API from LibSQL for future use for
offline-sync feature:
1. `wal_insert_begin` - begin WAL session by opening WAL read/write
transaction
2. `wal_insert_end` - finish WAL session by closing WAL transaction
opened by `wal_insert_begin` call
3. `wal_insert_frame` - insert frame `frame_no` with raw content `frame`
(WAL frame included)
For now any schema changes will not be reflected after
`wal_insert_frame` because `turso-db` do not re-parse schema without
need. I will fix this in follow up PR.
Reviewed-by: Pekka Enberg <penberg@iki.fi>
Closes#2231
## Background
The `balance_non_root` procedure can end up freeing a page if the pages
to be balanced can fit the required combined number of cells in less
pages, even if the page that triggered balancing is overfull. This can
then free the originally overfull pages, leaving a non-zero
`overflow_cells` on the in-mem representation of the page.
```rust
balance_non_root: page=305, overflow_cells=0
balance_non_root: page=304, overflow_cells=0
balance_non_root: page=302, overflow_cells=1
pre_edit_page(page=304, page_idx=0, new_cells=4, old_cells=1, cells_per_page_old=[1, 3, 9, 0, 0], cells_per_page_new=[4, 9, 9, 0, 0], cell_array_count=9)
edit_page start_old_cells=0 start_new_cells=0 number_new_cells=4 cell_array=9 end_old_cells=1 end_new_cells=4
pre_edit_page(page=305, page_idx=1, new_cells=4, old_cells=1, cells_per_page_old=[1, 3, 9, 0, 0], cells_per_page_new=[4, 9, 9, 0, 0], cell_array_count=9)
edit_page start_old_cells=2 start_new_cells=5 number_new_cells=4 cell_array=9 end_old_cells=3 end_new_cells=9
balance_non_root: sibling_count_new=2, sibling_count=3
// Custom assertion to demonstrate this:
thread 'main' panicked at core/storage/pager.rs:1127:29:
Pager::free_page: In memory page with id 302 has overflow cells
```
## Why is this a problem
Right now this is not an immediate problem, because we always allocate
brand new pages. However, in #2233 we begin to reuse pages from the
freelist for page allocation to improve performance and reduce database
size bloat. In that PR, the `balance_non_root` procedure will calculate
cell counts incorrectly in `edit_page()` and panic if: 1. a new
allocated page is taken from the freelist, 2. the page is still in
memory, and 3. and it still contains `overflow_cells`.
## Solution
Clear `page_contents.overflow_cells` when an in-memory page is freed.
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#2238
When compiling with features disabled, there are lots of clippy
warnings. This PR silences them.
For the utils file, I am using a bit of a hammer and just allowing
unused stuff in the whole file. Due to the box of utilities nature of
this file, it'll always be the case that things will be unused depending
on the feature-set.
There's no such thing as a read-only connection.
In a normal connection, you can have many attached databases. Some r/o,
some r/w.
To properly fix that, we also need to fix the OpenWrite opcode. Right
now we are passing a name, which is the name of the table. That
parameter is not used anywhere. That is also not what the SQLite opcode
specifies. Same as OpenRead, the p3 register should be the database
index.
With that change, we can - for now - pass the index 0, which is all we
support anyway, and then use that to test if we are r/o.
Reviewed-by: Preston Thorpe (@PThorpe92)
Closes#2232
There's no such thing as a read-only connection.
In a normal connection, you can have many attached databases. Some
r/o, some r/w.
To properly fix that, we also need to fix the OpenWrite opcode. Right
now we are passing a name, which is the name of the table. That
parameter is not used anywhere. That is also not what the SQLite opcode
specifies. Same as OpenRead, the p3 register should be the database
index.
With that change, we can - for now - pass the index 0, which is all
we support anyway, and then use that to test if we are r/o.
This PR fixes `wal_read_frame_raw` API
Before, implementation of raw read API read only page content - which is
not enough as we also need page_no and size_after fields from the
header. This PR fixes that and also make few adjustments in the
signatures.
Reviewed-by: Preston Thorpe (@PThorpe92)
Closes#2229
`BTreeTable::to_sql` makes us incompatible with SQLite by losing e.g. the original whitespace provided during the CREATE TABLE command.
For now let's fix our tests by regex-replacing every CREATE TABLE in
the entire repo to have exactly 1 space after the table name in the
CREATE TABLE statement.
A write txn can only start if the current snapshot held by writer is
consistent with the one in shared state
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#2204
