Commit Graph

9967 Commits

Author SHA1 Message Date
Pekka Enberg
a72b07e949 Merge 'Fix VDBE program abort' from Nikita Sivukhin
This PR add proper program abort in case of unfinished statement reset
and interruption.
Also, this PR makes rollback methods non-failing because otherwise of
their callers usually unclear (if rollback failed - what is the state of
statement/connection/transaction?)

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3591
2025-10-07 09:07:07 +03:00
Pekka Enberg
dacb8e3350 Merge 'Fix attach I/O error with in-memory databases' from Preston Thorpe
closes #3540

Closes #3602
2025-10-07 09:00:02 +03:00
Pekka Enberg
ce41bebaa8 Merge 'core/incremental: Fix re-insertion of data with same key' from Glauber Costa
There is currently a bug found in our materialized view implementation
that happens when we delete a row, and then re-insert another row with
the same primary key.
Our insert code needs to detect updates and generate a DELETE + INSERT.
But in this case, after the initial DELETE, the fresh insert generates
another delete.
We ended up with the wrong response for aggregations (and I am pretty
sure even filter-only views would manifest the bug as well), where
groups that should still be present just disappeared because of the
extra delete.
A new test case is added that fails without the fix.

Closes #3601
2025-10-07 08:47:38 +03:00
Pekka Enberg
e9be38c769 Merge 'Add MVCC checkpoint threshold pragma' from bit-aloo
closes: #3575

Closes #3604
2025-10-07 08:46:41 +03:00
Pekka Enberg
5f2beb1122 Merge 'core/incremental: Implement "is null" and "is not null" tests for view filter' from Glauber Costa
Just overlook on our side that they were not generated before.

Closes #3603
2025-10-07 08:45:45 +03:00
bit-aloo
afadb32c4c fmt fixes 2025-10-07 10:20:13 +05:30
bit-aloo
68b6ffe57c Implement mvcc_checkpoint_threshold pragma 2025-10-07 10:17:05 +05:30
bit-aloo
551dbf518e Add new mvcc_checkpoint_threshold pragma name 2025-10-07 10:17:05 +05:30
bit-aloo
fb5f5d9a90 Add MVCC checkpoint threshold APIs to Connection 2025-10-07 10:17:04 +05:30
bit-aloo
66c69461fb Add getter/setter for checkpoint threshold in LogicalLog
Wire threshold access through Storage

Add checkpoint threshold accessors to MvStore
2025-10-07 10:17:04 +05:30
Glauber Costa
e2694ff88b implement is null / is not null tests for mview filter
Just overlook on our side that they were not generated before.
2025-10-06 21:22:30 -05:00
PThorpe92
20d2ca55fe fix clippy warning 2025-10-06 21:43:48 -04:00
PThorpe92
addb9ef65b Add regression test for #3540 attach issue 2025-10-06 21:33:42 -04:00
PThorpe92
17da71ee3c Open db with proper IO when attaching database to fix #3540 2025-10-06 21:33:20 -04:00
Glauber Costa
beb44e8e8c fix mviews with re-insertion of data with the same key
There is currently a bug found in our materialized view implementation
that happens when we delete a row, and then re-insert another row with
the same primary key.

Our insert code needs to detect updates and generate a DELETE +
INSERT. But in this case, after the initial DELETE, the fresh insert
generates another delete.

We ended up with the wrong response for aggregations (and I am pretty
sure even filter-only views would manifest the bug as well), where
groups that should still be present just disappeared because of the
extra delete.

A new test case is added that fails without the fix.
2025-10-06 20:12:49 -05:00
Preston Thorpe
497808a40c Merge 'eliminate the need for another Once in Completion' from Pedro Muniz
I added the `Once` before so fix a bug, but it was a bit hackery. We can
`get_or_init` to achieve the same purpose, and the code becomes much
cleaner. `get_or_init` guarantees the init will happen only once as
well.

Reviewed-by: Preston Thorpe <preston@turso.tech>
Reviewed-by: bit-aloo (@Shourya742)

Closes #3578
2025-10-06 19:52:41 -04:00
Pekka Enberg
f8bdc02986 Merge 'Rename Completion methods' from Pedro Muniz
Reviewed-by: bit-aloo (@Shourya742)
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3579
2025-10-06 20:12:21 +03:00
Pekka Enberg
0134192cae Merge 'Top level examples' from Nikita Sivukhin
Closes #3599
2025-10-06 18:26:18 +03:00
Pere Diaz Bou
59c08c1062 Merge 'core/mvcc: implement PartialOrd for RowId' from Pere Diaz Bou
Closes #3597
2025-10-06 16:42:04 +02:00
Nikita Sivukhin
d2dab3d18c fix main workspace dir 2025-10-06 18:30:29 +04:00
Nikita Sivukhin
f0c7cff18c add README 2025-10-06 18:19:22 +04:00
pedrocarlo
2ce0e9db57 eliminate the need for another Once in Completion 2025-10-06 11:10:41 -03:00
pedrocarlo
5a7390735d rename Completion functions 2025-10-06 11:07:06 -03:00
Nikita Sivukhin
926fd25cdd move examples to the top-level directory 2025-10-06 18:01:54 +04:00
Nikita Sivukhin
e2f7310617 add explicit tracker for Txn cleanup necessary for statement 2025-10-06 17:51:43 +04:00
Pekka Enberg
0b9a5f331c Merge 'docs: Explain BEGIN CONCURRENT' from Pekka Enberg
Reviewed-by: bit-aloo (@Shourya742)

Closes #3555
2025-10-06 16:30:46 +03:00
Pekka Enberg
41d909e1fb Merge 'MVCC: do checkpoint writes in ascending order of rowid' from Jussi Saurio
For insert-heavy checkpoints this gives a much higher chance of using
the balance-quick subalgorithm instead of the more complex and slower
balance-nonroot.

Closes #3589
2025-10-06 16:30:19 +03:00
Pekka Enberg
3ac923b892 Merge ' core/mvcc: filter out seek results where is not same table_id ' from Pere Diaz Bou
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3596
2025-10-06 16:20:35 +03:00
Pere Diaz Bou
aed255d2e6 core/mvcc: implement PartialOrd for RowId 2025-10-06 13:43:42 +02:00
Pere Diaz Bou
fc7e1639a0 core/mvcc: filter out seek results where is not same table_id 2025-10-06 13:29:10 +02:00
Pere Diaz Bou
b9b9831d17 core/mvcc: test seek with empty table 2025-10-06 13:28:51 +02:00
Nikita Sivukhin
0ace1f9d90 fix code in order to not reset internal prepared statements created during DDL execution 2025-10-06 15:11:23 +04:00
Nikita Sivukhin
4877180784 fix clippy 2025-10-06 13:34:16 +04:00
Nikita Sivukhin
afe9a19496 add simple integration test 2025-10-06 13:30:05 +04:00
Nikita Sivukhin
a3ca5f6bf2 implement Drop for Statement 2025-10-06 13:27:42 +04:00
Nikita Sivukhin
48ca3864b8 properly abort statement in case of reset (when statement wasn't executed till completion) and interrupt 2025-10-06 13:22:26 +04:00
Nikita Sivukhin
8dae601fac make rollback non-failing method 2025-10-06 13:21:45 +04:00
Nikita Sivukhin
38d2630969 remove unnecessary SchemaLocked error
- lock() return error in case when another thread panicked while holding the same lock
- we better to just panic too in any such case
2025-10-06 12:15:15 +04:00
Jussi Saurio
fd9008f123 MVCC: do checkpoint writes in ascending order of rowid
For insert-heavy checkpoints this gives a much higher chance of using
the balance-quick subalgorithm instead of the more complex and slower
balance-nonroot.
2025-10-06 10:11:19 +03:00
Pekka Enberg
f12e0105e5 Merge 'core/io: Add completion group API for managing multiple I/O operations' from Pekka Enberg
Introduces a CompletionGroup abstraction that allows grouping multiple
I/O completions together for coordinated tracking and error handling.
This enables:
- Tracking completion status of multiple I/O operations as a group
- Detecting when all operations in a group have finished
- Aborting all operations in a group atomically
- Retrieving errors from any completion in the group
The implementation uses intrusive linked lists for efficient membership
tracking and atomic counters for outstanding operation counts. Each
completion can be linked to a group using the new .link() method.
This lays the groundwork for batch I/O operations and coordinated
transaction handling in the storage layer.

Reviewed-by: Pedro Muniz (@pedrocarlo)

Closes #3560
2025-10-06 09:19:50 +03:00
Pekka Enberg
be6f3d09ea core/storage: Switch checkpoint_inner() to completion group 2025-10-06 07:33:31 +03:00
Pekka Enberg
c27b167c6d core/io: Add completion group API for managing multiple I/O operations
Introduces a completion group abstraction that allows grouping multiple
I/O completions together for coordinated tracking and error handling.
This enables:

- Tracking completion status of multiple I/O operations as a group
- Detecting when all operations in a group have finished
- Aborting all operations in a group atomically
- Retrieving errors from any completion in the group

The implementation uses intrusive linked lists for efficient membership
tracking and atomic counters for outstanding operation counts. Each
completion can be linked to a group using the new .link() method.

This lays the groundwork for batch I/O operations and coordinated
transaction handling in the storage layer.
2025-10-06 07:33:31 +03:00
Jussi Saurio
aab7d989df Merge 'Simulator diff print' from Pedro Muniz
I was trying to debug MVCC simulator and database diffs, and this change
makes it much easier to visualize this. This PR is just a small quality
of life upgrade for debugability.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3548
2025-10-05 18:34:26 +03:00
Jussi Saurio
90632ad883 Merge 'Improve simulator cli' from bit-aloo
1. Moving manual CLI validation into Clap for safer argument handling.
2. Remove deprecated `with_ascii` flag from `PrettyFields` in logger
initialization.
3. Remove `log` and `env_logger` dependencies in favor of `tracing` from
simulator.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3533
2025-10-05 18:33:55 +03:00
Pekka Enberg
b52d6ab056 Merge 'core/mvcc: automatic logical log checkpointing on commit' from Pere Diaz Bou
On reaching 8 MiB checkpoint threshold we perform a blocking checkpoint
on the logical log. These changes modified how transaction_state is
tracked so that on a regular transaction we don't update it checkpoint's
state machine.
I wonder if checkpoint should stay inside commit's transaction locks or
like I did, where checkpoint happens right after committing transaction
but this happens on the same query during halt.

Closes #3565
2025-10-04 11:30:04 +03:00
Pekka Enberg
b063d0d41a Merge 'Don't panic if doing INSERT INTO ... SELECT rowid' from Jussi Saurio
Backport: 0.2
Closes #3567

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3572
2025-10-04 10:11:09 +03:00
Pekka Enberg
b81d45ae11 Merge 'remove dyn DatabaseStorage replace it with DatabaseFile' from Pedro Muniz
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3566
2025-10-04 10:08:33 +03:00
Pekka Enberg
50607607fa Merge 'Actually enforce uniqueness in create unique index' from Jussi Saurio
we just weren't doing it 🤡
Backport: 0.2
Closes #3568

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3571
2025-10-04 10:07:44 +03:00
Pere Diaz Bou
e3f2bbf90b core/mvcc: remove unnecessary fields in CommitState::Checkpoint 2025-10-03 22:27:03 +02:00
Pere Diaz Bou
c612a51df7 fmt 2025-10-03 22:24:38 +02:00