Commit Graph

5987 Commits

Author SHA1 Message Date
Zaid Humayun
2f0ff89e28 resolved jussi's comment https://github.com/tursodatabase/turso/pull/1966#discussion_r2201864782
this commit removes !page.is_loaded() from header_accessor cause it's not required
2025-07-12 09:44:31 +05:30
Zaid Humayun
90a5a53b0e Added Async Header Accessor API's
This commit introduces async header accessor API's in addition to the sync ones. Allows gradual porting instead of one big shot PR.
2025-07-12 09:38:18 +05:30
Pekka Enberg
f28e2314b1 Merge 'Simulator - only output color on terminal' from Mikaël Francoeur
This inhibits ANSI color codes on non-TTY stdouts. It should fix logs
with mangled color codes such as
https://github.com/tursodatabase/turso/issues/2046.
## Result
On this branch:
<img width="1492" height="421" alt="image" src="https://github.com/user-
attachments/assets/1d281203-bda1-41c6-944e-f10dac16834e" />
If I don't pipe or redirect, on this branch, the color is displayed.
On main:
<img width="1496" height="438" alt="image" src="https://github.com/user-
attachments/assets/3844edac-dead-4655-96fb-df6ab5353021" />

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2052
2025-07-11 21:18:17 +03:00
Mikaël Francoeur
ed2de42048 only output color on terminal 2025-07-11 10:57:01 -04:00
Jussi Saurio
0009683566 Merge 'btree/balance/validation: fix divider cell insert validation' from Jussi Saurio
Closes #2047
the validation code was assuming that:
- if the parent has overflow cells after inserting a divider cell
- the exact divider we are validating MUST be in those overflow cells
However, this is not necessarily the case. Imagine:
- First divider gets inserted at index `n`. It is too large to fit, so
it gets pushed to `parent.overflow_cells()`. Parent usable space does
not decrease.
- Second divider gets inserted at index `n+1`. It is smaller, so it
still fits in usable space.
Hence:
Provide information to the validation function about whether the
inserted cell overflowed, and use that to find the left pointer and
assert accordingly.

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #2050
2025-07-11 15:09:19 +03:00
Pekka Enberg
7b646da82c Merge 'Btree: more balance docs' from Jussi Saurio
Continuation from #2031 -- some variable extractions and comments

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #2035
2025-07-11 15:06:45 +03:00
Pekka Enberg
e90ff42484 antithesis: Fix first_setup.py to commit both transactions 2025-07-11 13:12:26 +03:00
Jussi Saurio
c81522c20f btree/balance/validation: fix divider cell insert validation
the validation code was assuming that:

- if the parent has overflow cells after a inserting divider cell
- the exact divider we are validating MUST be in those overflow cells

However, this is not necessarily the case. Imagine:

- First divider gets inserted at index `n`. It is too large to fit,
  so it gets pushed to `parent.overflow_cells()`. Parent usable space
  does not decrease.
- Second divider gets inserted at index `n+1`. It is smaller, so it
  still fits in usable space.

Hence:

Provide information to the validation function about whether the inserted
cell overflowed, and use that to find the left pointer and assert accordingly.
2025-07-11 11:21:05 +03:00
Jussi Saurio
37176160f9 Merge 'btree/balance/validation: fix use-after-free in rightmost ptr validation' from Jussi Saurio
We can use `right_page_id` directly to perform the validation instead of
carrying a raw pointer around which might be invalidated by the time we
do the validation.
Closes #2036
Closes #1991

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #2038
2025-07-11 10:24:37 +03:00
Pekka Enberg
ea7a5c5d4e Merge 'vdbe: fix some issues with min() and max() and add ignored fuzz test' from Jussi Saurio
Removes the error prints described in #2033 but doesn't fix the issue.
Instead as a result of this PR I opened
https://github.com/tursodatabase/turso/issues/2040 and
https://github.com/tursodatabase/turso/issues/2041 after I added a fuzz
test 😂
we _need_ to fuzz _everything_.

Closes #2039
2025-07-11 09:52:56 +03:00
Jussi Saurio
897f59fab1 test/fuzz: add ignored fuzz test for min()/max() - ignored because of bugs 2025-07-10 21:02:57 +03:00
Jussi Saurio
b015fabb26 vdbe: fix panic when first value added to min()/max() accumulator is null 2025-07-10 21:02:57 +03:00
Jussi Saurio
63c5698050 vdbe: remove error prints from min()/max() and simplify 2025-07-10 21:02:57 +03:00
Pekka Enberg
04c74fb794 Merge 'github: Update to newest Nyrkiö Github action' from Henrik Ingo
I noticed on instance of the nyrkio github action was still using the
old name. Updating to nyrkio/change-detection@HEAD. All other entries
are already updaeted,

Closes #2037
2025-07-10 19:41:57 +03:00
Pekka Enberg
7c70e8274f antithesis: Run experiments for 8 hours
Eric from Antithesis pointed out that we can still find more states by
running for longer, so let's try that.
2025-07-10 19:37:45 +03:00
Jussi Saurio
99df69f603 btree/balance/validation: fix use-after-free of rightmost ptr validation
We can use `right_page_id` directly to perform the validation instead of
carrying a raw pointer around which might be invalidated by the time we
do the validation.
2025-07-10 18:23:54 +03:00
Henrik Ingo
8d9596ea41 Workflows: Update to newest Nyrkiö Github Action
I noticed on instance of the nyrkio github action was still
using the old name. Updating to nyrkio/change-detection@HEAD.
All other entries are already updaeted,
2025-07-10 17:40:34 +03:00
Pekka Enberg
9ddc77d54d Merge 'core/translate: Fix "misuse of aggregate function" error message' from Pekka Enberg
```
sqlite> CREATE TABLE test1(f1, f2);
sqlite> SELECT SUM(min(f1)) FROM test1;
Parse error: misuse of aggregate function min()
  SELECT SUM(min(f1)) FROM test1;
             ^--- error here
```
Spotted by SQLite TCL tests.

Closes #2032
2025-07-10 16:35:30 +03:00
Jussi Saurio
a403e55319 btree: add comment about when left-to-right size balancing is stopped 2025-07-10 15:52:18 +03:00
Jussi Saurio
bc328f9738 btree: one more is_last_sibling doc variable 2025-07-10 15:39:22 +03:00
Jussi Saurio
fc27c08e11 clippy 2025-07-10 15:36:46 +03:00
Jussi Saurio
fba05b1998 btree: add named range variables to make cell movement double-pass clearer 2025-07-10 15:34:41 +03:00
Jussi Saurio
8f1109692f btree: replace a bunch of 'count-1' conditions with is_last_sibling variables 2025-07-10 15:31:53 +03:00
Jussi Saurio
a81d81685f btree: rename divider_cells to divider_cell_payloads for clarity 2025-07-10 15:29:01 +03:00
Jussi Saurio
5d0b410e70 btree: rename constant to mention siblings 2025-07-10 15:28:11 +03:00
Jussi Saurio
78a249c6d0 btree: add MAX_SIBLING_PAGES_TO_BALANCE constant and use it 2025-07-10 15:27:37 +03:00
Jussi Saurio
6ff13113ce btree: add MAX_NEW_PAGES_AFTER_BALANCE constant and use it 2025-07-10 15:24:32 +03:00
Jussi Saurio
10d301c53c btree: use size-related new constants instead of literal numbers 2025-07-10 15:21:00 +03:00
Jussi Saurio
59b4f1310b sqlite3_ondisk: add some size constants 2025-07-10 15:16:37 +03:00
Jussi Saurio
925a252815 Merge 'btree: Improve balance non root docs' from Jussi Saurio
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #2031
2025-07-10 15:07:18 +03:00
Jussi Saurio
0b8c5f7c91 btree/balance: extra doc context for CellArray::cell_payloads 2025-07-10 15:06:27 +03:00
Jussi Saurio
475bced4f7 btree/balance: remove obsolete todo 2025-07-10 14:58:00 +03:00
Jussi Saurio
0316b5a517 btree/balance: rename CellArray::cell_data to cell_payloads 2025-07-10 14:57:45 +03:00
Jussi Saurio
0d973d78a9 btree/balance: add a diagram about divider cell assignment and some comments 2025-07-10 14:56:59 +03:00
Pekka Enberg
6620ca954d testing/sqlite3: Fix NULL handling in tester.tcl 2025-07-10 14:41:33 +03:00
Pekka Enberg
f24e254ec6 core/translate: Fix "misuse of aggregate function" error message
```
sqlite> CREATE TABLE test1(f1, f2);
sqlite> SELECT SUM(min(f1)) FROM test1;
Parse error: misuse of aggregate function min()
  SELECT SUM(min(f1)) FROM test1;
             ^--- error here
```

Spotted by SQLite TCL tests.
2025-07-10 14:29:59 +03:00
Pekka Enberg
6749af7037 Merge 'core/translate: Return error if SELECT needs tables and there are none' from Mikaël Francoeur
Fixes https://github.com/tursodatabase/turso/issues/1972

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2023
2025-07-10 14:02:57 +03:00
Pekka Enberg
7a259957ac Merge 'properly set last_checksum after recovering wal' from Pere Diaz Bou
We store `last_checksum` to do cumulative checksumming. After reading
wal for recovery, we didn't set last checksum properly in case there
were no frames so this cause us to not initialize last_checksum
properly.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2030
2025-07-10 13:54:15 +03:00
Jussi Saurio
610b743f0d btree/balance: rename CellArray::cell_count to CellArray::cell_count_up_to_page 2025-07-10 13:51:56 +03:00
Jussi Saurio
924482981c btree/balance: rename CellArray::cell_size to CellArray::cell_size_bytes 2025-07-10 13:50:14 +03:00
Jussi Saurio
68cd948056 btree/balance: add extra documentation for page update dual pass 2025-07-10 13:47:08 +03:00
Pekka Enberg
1333fc884c github: Make Antithesis email a secret
I am adding a mailing list address that I prefer not to share.
2025-07-10 13:42:26 +03:00
Pekka Enberg
5a961c0837 Merge 'antithesis: Fix transaction management' from Pekka Enberg
Commit 5216e67d ("bindings/python: Start transaction implicitly in
execute()") fixed transaction management in Python bindings, which means
we now need to execute explicit commit().

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2028
2025-07-10 13:38:12 +03:00
Pere Diaz Bou
832f9fb8a8 clippy 2025-07-10 12:23:33 +02:00
Jussi Saurio
3fc51ed4d9 btree/balance: rename leaf to is_leaf 2025-07-10 13:16:14 +03:00
Jussi Saurio
d88bbd488f btree/balance: rename leaf_data to is_table_leaf 2025-07-10 13:15:29 +03:00
Pere Diaz Bou
9258d33d8b properly set last_checksum after recovering wal
We store `last_checksum` to do cumulative checksumming. After reading
wal for recovery, we didn't set last checksum properly in case there
were no frames so this cause us to not initialize last_checksum
properly.
2025-07-10 12:15:00 +02:00
Jussi Saurio
b306550a69 format 2025-07-10 13:14:57 +03:00
Jussi Saurio
5ef0127409 btree/balance: rename count_cells_in_old_pages to old_cell_count_per_page_cumulative 2025-07-10 13:14:18 +03:00
Jussi Saurio
c31ee0e628 btree/balance: rename number_of_cells_per_page to cell_count_per_page_cumulative 2025-07-10 13:12:17 +03:00