Commit Graph

6011 Commits

Author SHA1 Message Date
Krishna Vishal
ef147181c2 Working version of the incremental column and "optimal" record
compare functions. Now we optimize them
2025-07-14 03:28:54 +05:30
Krishna Vishal
692f0413eb Stash 2025-07-14 03:28:54 +05:30
Krishna Vishal
0b1ed44c1f Add optimized index record compare methods
compare_records_int
compare_records_string
compare_records_generic

comapre_records_generic will still be more efficient than compare-
_immutable because it deserializes the record column by column
2025-07-14 03:28:54 +05:30
Krishna Vishal
c7aa3c3d93 Fix btree to invalidate RecordCursor
Use `read_value` instead of `deserialize_column_data`
Add `sqlite_int_float_compare` which takes care of out of range
floats
2025-07-14 03:28:54 +05:30
Krishna Vishal
515712b7f2 Fix sorter 2025-07-14 03:28:54 +05:30
Krishna Vishal
601540af6e Make OP_column do on demand serialization baby! 2025-07-14 03:28:54 +05:30
Krishna Vishal
2323763a4f Integrate incremental column parsing into btree.rs 2025-07-14 03:28:54 +05:30
Krishna Vishal
6c5b95a8d7 Add get_values methods to ImmutableRecord 2025-07-14 03:28:54 +05:30
Krishna Vishal
35fa9b368c Decouple Value parsing and Record loading.
Introduced `RecordCursor`, using which we can parse the record
header incrementally.
2025-07-14 03:28:54 +05:30
Krishna Vishal
180bcc7b60 Add incremental and on-demand parsing of ImmutableRecord.
First step at resolving the currently wasteful eager parsing.
2025-07-14 03:28:54 +05:30
Pekka Enberg
c454feeba4 Turso 0.1.2-pre.3 2025-07-13 12:55:02 +03:00
Jussi Saurio
5f6a7965cb Merge 'Fix clippy errors for Rust 1.88.0' from Nils Koch
I am currently working on upgrading Rust to the latest version
(https://github.com/tursodatabase/limbo/pull/1807). Initially, I wanted
to address all Clippy errors in that PR, but I decided to move them into
a separate one to make it easier to review.
**Note:** In the latest version of Clippy, it is considered an error to
place variables after a comma in string formatting macros (e.g.,
`format!`), rather than inside the `{}` placeholders. That's where most
of the errors are coming from.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1827
2025-07-12 19:36:05 +03:00
Jussi Saurio
a48b6d049a Another post-rebase clippy round with 1.88.0 2025-07-12 19:10:56 +03:00
Nils Koch
1a91966c7e fix clippy errors for rust 1.88.0 (manual fix) 2025-07-12 18:58:55 +03:00
Nils Koch
828d4f5016 fix clippy errors for rust 1.88.0 (auto fix) 2025-07-12 18:58:41 +03:00
Jussi Saurio
b68aaebe50 Merge 'sim: return LimboError::Busy when busy, instead of looping forever' from Jussi Saurio
Although concurrency doesn't work yet (#2059), we do want to support
concurrency, and one good first step is not to make the simulator hang
forever once any connection gets a `Busy` result from the VDBE
Closes #2060

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #2061
2025-07-12 10:34:21 +03:00
Jussi Saurio
cb16301157 sim: return LimboError::Busy when busy, instead of looping forever 2025-07-12 10:24:49 +03:00
Pekka Enberg
416ab28c2c Merge 'parse_schema_rows optimizations' from Levy A.
- Also added a benchmark for opening databases, the main thing that is
slowing `Database::open_file` is `parse_schema_rows`.
- `to_uppercase` was being called multiple times, leaving a relevant
mark on stack traces due to multiple allocations. `make_ascii_upper`
reuses the memory and is faster due to not handling unicode characters
(still compatible with sqlite).
- Do direct btree calls instead of creating a program for updating
`Schema` with `Schema::make_from_btree`.
- Faster type substr comparison using fixed size `u8` slices.
<img width="952" height="507" alt="image" src="https://github.com/user-
attachments/assets/0d0c52ff-05a1-431e-a93d-e333b53c0bb8" />

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2042
2025-07-12 09:11:20 +03:00
Pekka Enberg
f28e2314b1 Merge 'Simulator - only output color on terminal' from Mikaël Francoeur
This inhibits ANSI color codes on non-TTY stdouts. It should fix logs
with mangled color codes such as
https://github.com/tursodatabase/turso/issues/2046.
## Result
On this branch:
<img width="1492" height="421" alt="image" src="https://github.com/user-
attachments/assets/1d281203-bda1-41c6-944e-f10dac16834e" />
If I don't pipe or redirect, on this branch, the color is displayed.
On main:
<img width="1496" height="438" alt="image" src="https://github.com/user-
attachments/assets/3844edac-dead-4655-96fb-df6ab5353021" />

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2052
2025-07-11 21:18:17 +03:00
Levy A.
a1e418c999 fix tests 2025-07-11 15:04:28 -03:00
Levy A.
b1341113d7 clippy 2025-07-11 15:04:28 -03:00
Levy A.
b008c787b7 faster type substr comparison 2025-07-11 15:04:28 -03:00
Levy A.
c300a01120 fix: add space between column name and type 2025-07-11 15:04:28 -03:00
Levy A.
cc17211189 direct btree calls 2025-07-11 15:04:28 -03:00
Levy A.
c145577bce fix: use ty_str for SQL conversion 2025-07-11 15:04:28 -03:00
Levy A.
a479d0d5e8 prevent calling to_uppercase 2025-07-11 15:04:28 -03:00
Levy A.
9c0e850622 add connection benchmark 2025-07-11 15:04:24 -03:00
Mikaël Francoeur
ed2de42048 only output color on terminal 2025-07-11 10:57:01 -04:00
Jussi Saurio
0009683566 Merge 'btree/balance/validation: fix divider cell insert validation' from Jussi Saurio
Closes #2047
the validation code was assuming that:
- if the parent has overflow cells after inserting a divider cell
- the exact divider we are validating MUST be in those overflow cells
However, this is not necessarily the case. Imagine:
- First divider gets inserted at index `n`. It is too large to fit, so
it gets pushed to `parent.overflow_cells()`. Parent usable space does
not decrease.
- Second divider gets inserted at index `n+1`. It is smaller, so it
still fits in usable space.
Hence:
Provide information to the validation function about whether the
inserted cell overflowed, and use that to find the left pointer and
assert accordingly.

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #2050
2025-07-11 15:09:19 +03:00
Pekka Enberg
7b646da82c Merge 'Btree: more balance docs' from Jussi Saurio
Continuation from #2031 -- some variable extractions and comments

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #2035
2025-07-11 15:06:45 +03:00
Pekka Enberg
e90ff42484 antithesis: Fix first_setup.py to commit both transactions 2025-07-11 13:12:26 +03:00
Jussi Saurio
c81522c20f btree/balance/validation: fix divider cell insert validation
the validation code was assuming that:

- if the parent has overflow cells after a inserting divider cell
- the exact divider we are validating MUST be in those overflow cells

However, this is not necessarily the case. Imagine:

- First divider gets inserted at index `n`. It is too large to fit,
  so it gets pushed to `parent.overflow_cells()`. Parent usable space
  does not decrease.
- Second divider gets inserted at index `n+1`. It is smaller, so it
  still fits in usable space.

Hence:

Provide information to the validation function about whether the inserted
cell overflowed, and use that to find the left pointer and assert accordingly.
2025-07-11 11:21:05 +03:00
Jussi Saurio
37176160f9 Merge 'btree/balance/validation: fix use-after-free in rightmost ptr validation' from Jussi Saurio
We can use `right_page_id` directly to perform the validation instead of
carrying a raw pointer around which might be invalidated by the time we
do the validation.
Closes #2036
Closes #1991

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #2038
2025-07-11 10:24:37 +03:00
Pekka Enberg
ea7a5c5d4e Merge 'vdbe: fix some issues with min() and max() and add ignored fuzz test' from Jussi Saurio
Removes the error prints described in #2033 but doesn't fix the issue.
Instead as a result of this PR I opened
https://github.com/tursodatabase/turso/issues/2040 and
https://github.com/tursodatabase/turso/issues/2041 after I added a fuzz
test 😂
we _need_ to fuzz _everything_.

Closes #2039
2025-07-11 09:52:56 +03:00
Jussi Saurio
897f59fab1 test/fuzz: add ignored fuzz test for min()/max() - ignored because of bugs 2025-07-10 21:02:57 +03:00
Jussi Saurio
b015fabb26 vdbe: fix panic when first value added to min()/max() accumulator is null 2025-07-10 21:02:57 +03:00
Jussi Saurio
63c5698050 vdbe: remove error prints from min()/max() and simplify 2025-07-10 21:02:57 +03:00
Pekka Enberg
04c74fb794 Merge 'github: Update to newest Nyrkiö Github action' from Henrik Ingo
I noticed on instance of the nyrkio github action was still using the
old name. Updating to nyrkio/change-detection@HEAD. All other entries
are already updaeted,

Closes #2037
2025-07-10 19:41:57 +03:00
Pekka Enberg
7c70e8274f antithesis: Run experiments for 8 hours
Eric from Antithesis pointed out that we can still find more states by
running for longer, so let's try that.
2025-07-10 19:37:45 +03:00
Jussi Saurio
99df69f603 btree/balance/validation: fix use-after-free of rightmost ptr validation
We can use `right_page_id` directly to perform the validation instead of
carrying a raw pointer around which might be invalidated by the time we
do the validation.
2025-07-10 18:23:54 +03:00
Henrik Ingo
8d9596ea41 Workflows: Update to newest Nyrkiö Github Action
I noticed on instance of the nyrkio github action was still
using the old name. Updating to nyrkio/change-detection@HEAD.
All other entries are already updaeted,
2025-07-10 17:40:34 +03:00
Pekka Enberg
9ddc77d54d Merge 'core/translate: Fix "misuse of aggregate function" error message' from Pekka Enberg
```
sqlite> CREATE TABLE test1(f1, f2);
sqlite> SELECT SUM(min(f1)) FROM test1;
Parse error: misuse of aggregate function min()
  SELECT SUM(min(f1)) FROM test1;
             ^--- error here
```
Spotted by SQLite TCL tests.

Closes #2032
2025-07-10 16:35:30 +03:00
Jussi Saurio
a403e55319 btree: add comment about when left-to-right size balancing is stopped 2025-07-10 15:52:18 +03:00
Jussi Saurio
bc328f9738 btree: one more is_last_sibling doc variable 2025-07-10 15:39:22 +03:00
Jussi Saurio
fc27c08e11 clippy 2025-07-10 15:36:46 +03:00
Jussi Saurio
fba05b1998 btree: add named range variables to make cell movement double-pass clearer 2025-07-10 15:34:41 +03:00
Jussi Saurio
8f1109692f btree: replace a bunch of 'count-1' conditions with is_last_sibling variables 2025-07-10 15:31:53 +03:00
Jussi Saurio
a81d81685f btree: rename divider_cells to divider_cell_payloads for clarity 2025-07-10 15:29:01 +03:00
Jussi Saurio
5d0b410e70 btree: rename constant to mention siblings 2025-07-10 15:28:11 +03:00
Jussi Saurio
78a249c6d0 btree: add MAX_SIBLING_PAGES_TO_BALANCE constant and use it 2025-07-10 15:27:37 +03:00