Commit Graph

3271 Commits

Author SHA1 Message Date
Krishna Vishal
ef147181c2 Working version of the incremental column and "optimal" record
compare functions. Now we optimize them
2025-07-14 03:28:54 +05:30
Krishna Vishal
692f0413eb Stash 2025-07-14 03:28:54 +05:30
Krishna Vishal
0b1ed44c1f Add optimized index record compare methods
compare_records_int
compare_records_string
compare_records_generic

comapre_records_generic will still be more efficient than compare-
_immutable because it deserializes the record column by column
2025-07-14 03:28:54 +05:30
Krishna Vishal
c7aa3c3d93 Fix btree to invalidate RecordCursor
Use `read_value` instead of `deserialize_column_data`
Add `sqlite_int_float_compare` which takes care of out of range
floats
2025-07-14 03:28:54 +05:30
Krishna Vishal
515712b7f2 Fix sorter 2025-07-14 03:28:54 +05:30
Krishna Vishal
601540af6e Make OP_column do on demand serialization baby! 2025-07-14 03:28:54 +05:30
Krishna Vishal
2323763a4f Integrate incremental column parsing into btree.rs 2025-07-14 03:28:54 +05:30
Krishna Vishal
6c5b95a8d7 Add get_values methods to ImmutableRecord 2025-07-14 03:28:54 +05:30
Krishna Vishal
35fa9b368c Decouple Value parsing and Record loading.
Introduced `RecordCursor`, using which we can parse the record
header incrementally.
2025-07-14 03:28:54 +05:30
Krishna Vishal
180bcc7b60 Add incremental and on-demand parsing of ImmutableRecord.
First step at resolving the currently wasteful eager parsing.
2025-07-14 03:28:54 +05:30
Jussi Saurio
a48b6d049a Another post-rebase clippy round with 1.88.0 2025-07-12 19:10:56 +03:00
Nils Koch
1a91966c7e fix clippy errors for rust 1.88.0 (manual fix) 2025-07-12 18:58:55 +03:00
Nils Koch
828d4f5016 fix clippy errors for rust 1.88.0 (auto fix) 2025-07-12 18:58:41 +03:00
Pekka Enberg
416ab28c2c Merge 'parse_schema_rows optimizations' from Levy A.
- Also added a benchmark for opening databases, the main thing that is
slowing `Database::open_file` is `parse_schema_rows`.
- `to_uppercase` was being called multiple times, leaving a relevant
mark on stack traces due to multiple allocations. `make_ascii_upper`
reuses the memory and is faster due to not handling unicode characters
(still compatible with sqlite).
- Do direct btree calls instead of creating a program for updating
`Schema` with `Schema::make_from_btree`.
- Faster type substr comparison using fixed size `u8` slices.
<img width="952" height="507" alt="image" src="https://github.com/user-
attachments/assets/0d0c52ff-05a1-431e-a93d-e333b53c0bb8" />

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2042
2025-07-12 09:11:20 +03:00
Levy A.
a1e418c999 fix tests 2025-07-11 15:04:28 -03:00
Levy A.
b1341113d7 clippy 2025-07-11 15:04:28 -03:00
Levy A.
b008c787b7 faster type substr comparison 2025-07-11 15:04:28 -03:00
Levy A.
c300a01120 fix: add space between column name and type 2025-07-11 15:04:28 -03:00
Levy A.
cc17211189 direct btree calls 2025-07-11 15:04:28 -03:00
Levy A.
c145577bce fix: use ty_str for SQL conversion 2025-07-11 15:04:28 -03:00
Levy A.
a479d0d5e8 prevent calling to_uppercase 2025-07-11 15:04:28 -03:00
Levy A.
9c0e850622 add connection benchmark 2025-07-11 15:04:24 -03:00
Jussi Saurio
0009683566 Merge 'btree/balance/validation: fix divider cell insert validation' from Jussi Saurio
Closes #2047
the validation code was assuming that:
- if the parent has overflow cells after inserting a divider cell
- the exact divider we are validating MUST be in those overflow cells
However, this is not necessarily the case. Imagine:
- First divider gets inserted at index `n`. It is too large to fit, so
it gets pushed to `parent.overflow_cells()`. Parent usable space does
not decrease.
- Second divider gets inserted at index `n+1`. It is smaller, so it
still fits in usable space.
Hence:
Provide information to the validation function about whether the
inserted cell overflowed, and use that to find the left pointer and
assert accordingly.

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #2050
2025-07-11 15:09:19 +03:00
Pekka Enberg
7b646da82c Merge 'Btree: more balance docs' from Jussi Saurio
Continuation from #2031 -- some variable extractions and comments

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #2035
2025-07-11 15:06:45 +03:00
Jussi Saurio
c81522c20f btree/balance/validation: fix divider cell insert validation
the validation code was assuming that:

- if the parent has overflow cells after a inserting divider cell
- the exact divider we are validating MUST be in those overflow cells

However, this is not necessarily the case. Imagine:

- First divider gets inserted at index `n`. It is too large to fit,
  so it gets pushed to `parent.overflow_cells()`. Parent usable space
  does not decrease.
- Second divider gets inserted at index `n+1`. It is smaller, so it
  still fits in usable space.

Hence:

Provide information to the validation function about whether the inserted
cell overflowed, and use that to find the left pointer and assert accordingly.
2025-07-11 11:21:05 +03:00
Jussi Saurio
37176160f9 Merge 'btree/balance/validation: fix use-after-free in rightmost ptr validation' from Jussi Saurio
We can use `right_page_id` directly to perform the validation instead of
carrying a raw pointer around which might be invalidated by the time we
do the validation.
Closes #2036
Closes #1991

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #2038
2025-07-11 10:24:37 +03:00
Jussi Saurio
b015fabb26 vdbe: fix panic when first value added to min()/max() accumulator is null 2025-07-10 21:02:57 +03:00
Jussi Saurio
63c5698050 vdbe: remove error prints from min()/max() and simplify 2025-07-10 21:02:57 +03:00
Jussi Saurio
99df69f603 btree/balance/validation: fix use-after-free of rightmost ptr validation
We can use `right_page_id` directly to perform the validation instead of
carrying a raw pointer around which might be invalidated by the time we
do the validation.
2025-07-10 18:23:54 +03:00
Pekka Enberg
9ddc77d54d Merge 'core/translate: Fix "misuse of aggregate function" error message' from Pekka Enberg
```
sqlite> CREATE TABLE test1(f1, f2);
sqlite> SELECT SUM(min(f1)) FROM test1;
Parse error: misuse of aggregate function min()
  SELECT SUM(min(f1)) FROM test1;
             ^--- error here
```
Spotted by SQLite TCL tests.

Closes #2032
2025-07-10 16:35:30 +03:00
Jussi Saurio
a403e55319 btree: add comment about when left-to-right size balancing is stopped 2025-07-10 15:52:18 +03:00
Jussi Saurio
bc328f9738 btree: one more is_last_sibling doc variable 2025-07-10 15:39:22 +03:00
Jussi Saurio
fc27c08e11 clippy 2025-07-10 15:36:46 +03:00
Jussi Saurio
fba05b1998 btree: add named range variables to make cell movement double-pass clearer 2025-07-10 15:34:41 +03:00
Jussi Saurio
8f1109692f btree: replace a bunch of 'count-1' conditions with is_last_sibling variables 2025-07-10 15:31:53 +03:00
Jussi Saurio
a81d81685f btree: rename divider_cells to divider_cell_payloads for clarity 2025-07-10 15:29:01 +03:00
Jussi Saurio
5d0b410e70 btree: rename constant to mention siblings 2025-07-10 15:28:11 +03:00
Jussi Saurio
78a249c6d0 btree: add MAX_SIBLING_PAGES_TO_BALANCE constant and use it 2025-07-10 15:27:37 +03:00
Jussi Saurio
6ff13113ce btree: add MAX_NEW_PAGES_AFTER_BALANCE constant and use it 2025-07-10 15:24:32 +03:00
Jussi Saurio
10d301c53c btree: use size-related new constants instead of literal numbers 2025-07-10 15:21:00 +03:00
Jussi Saurio
59b4f1310b sqlite3_ondisk: add some size constants 2025-07-10 15:16:37 +03:00
Jussi Saurio
925a252815 Merge 'btree: Improve balance non root docs' from Jussi Saurio
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #2031
2025-07-10 15:07:18 +03:00
Jussi Saurio
0b8c5f7c91 btree/balance: extra doc context for CellArray::cell_payloads 2025-07-10 15:06:27 +03:00
Jussi Saurio
475bced4f7 btree/balance: remove obsolete todo 2025-07-10 14:58:00 +03:00
Jussi Saurio
0316b5a517 btree/balance: rename CellArray::cell_data to cell_payloads 2025-07-10 14:57:45 +03:00
Jussi Saurio
0d973d78a9 btree/balance: add a diagram about divider cell assignment and some comments 2025-07-10 14:56:59 +03:00
Pekka Enberg
f24e254ec6 core/translate: Fix "misuse of aggregate function" error message
```
sqlite> CREATE TABLE test1(f1, f2);
sqlite> SELECT SUM(min(f1)) FROM test1;
Parse error: misuse of aggregate function min()
  SELECT SUM(min(f1)) FROM test1;
             ^--- error here
```

Spotted by SQLite TCL tests.
2025-07-10 14:29:59 +03:00
Pekka Enberg
6749af7037 Merge 'core/translate: Return error if SELECT needs tables and there are none' from Mikaël Francoeur
Fixes https://github.com/tursodatabase/turso/issues/1972

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2023
2025-07-10 14:02:57 +03:00
Pekka Enberg
7a259957ac Merge 'properly set last_checksum after recovering wal' from Pere Diaz Bou
We store `last_checksum` to do cumulative checksumming. After reading
wal for recovery, we didn't set last checksum properly in case there
were no frames so this cause us to not initialize last_checksum
properly.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2030
2025-07-10 13:54:15 +03:00
Jussi Saurio
610b743f0d btree/balance: rename CellArray::cell_count to CellArray::cell_count_up_to_page 2025-07-10 13:51:56 +03:00