Commit Graph

5547 Commits

Author SHA1 Message Date
Preston Thorpe
497808a40c Merge 'eliminate the need for another Once in Completion' from Pedro Muniz
I added the `Once` before so fix a bug, but it was a bit hackery. We can
`get_or_init` to achieve the same purpose, and the code becomes much
cleaner. `get_or_init` guarantees the init will happen only once as
well.

Reviewed-by: Preston Thorpe <preston@turso.tech>
Reviewed-by: bit-aloo (@Shourya742)

Closes #3578
2025-10-06 19:52:41 -04:00
Pekka Enberg
f8bdc02986 Merge 'Rename Completion methods' from Pedro Muniz
Reviewed-by: bit-aloo (@Shourya742)
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3579
2025-10-06 20:12:21 +03:00
Pere Diaz Bou
59c08c1062 Merge 'core/mvcc: implement PartialOrd for RowId' from Pere Diaz Bou
Closes #3597
2025-10-06 16:42:04 +02:00
pedrocarlo
2ce0e9db57 eliminate the need for another Once in Completion 2025-10-06 11:10:41 -03:00
pedrocarlo
5a7390735d rename Completion functions 2025-10-06 11:07:06 -03:00
Pekka Enberg
41d909e1fb Merge 'MVCC: do checkpoint writes in ascending order of rowid' from Jussi Saurio
For insert-heavy checkpoints this gives a much higher chance of using
the balance-quick subalgorithm instead of the more complex and slower
balance-nonroot.

Closes #3589
2025-10-06 16:30:19 +03:00
Pere Diaz Bou
aed255d2e6 core/mvcc: implement PartialOrd for RowId 2025-10-06 13:43:42 +02:00
Pere Diaz Bou
fc7e1639a0 core/mvcc: filter out seek results where is not same table_id 2025-10-06 13:29:10 +02:00
Pere Diaz Bou
b9b9831d17 core/mvcc: test seek with empty table 2025-10-06 13:28:51 +02:00
Jussi Saurio
fd9008f123 MVCC: do checkpoint writes in ascending order of rowid
For insert-heavy checkpoints this gives a much higher chance of using
the balance-quick subalgorithm instead of the more complex and slower
balance-nonroot.
2025-10-06 10:11:19 +03:00
Pekka Enberg
be6f3d09ea core/storage: Switch checkpoint_inner() to completion group 2025-10-06 07:33:31 +03:00
Pekka Enberg
c27b167c6d core/io: Add completion group API for managing multiple I/O operations
Introduces a completion group abstraction that allows grouping multiple
I/O completions together for coordinated tracking and error handling.
This enables:

- Tracking completion status of multiple I/O operations as a group
- Detecting when all operations in a group have finished
- Aborting all operations in a group atomically
- Retrieving errors from any completion in the group

The implementation uses intrusive linked lists for efficient membership
tracking and atomic counters for outstanding operation counts. Each
completion can be linked to a group using the new .link() method.

This lays the groundwork for batch I/O operations and coordinated
transaction handling in the storage layer.
2025-10-06 07:33:31 +03:00
Pekka Enberg
b52d6ab056 Merge 'core/mvcc: automatic logical log checkpointing on commit' from Pere Diaz Bou
On reaching 8 MiB checkpoint threshold we perform a blocking checkpoint
on the logical log. These changes modified how transaction_state is
tracked so that on a regular transaction we don't update it checkpoint's
state machine.
I wonder if checkpoint should stay inside commit's transaction locks or
like I did, where checkpoint happens right after committing transaction
but this happens on the same query during halt.

Closes #3565
2025-10-04 11:30:04 +03:00
Pekka Enberg
b063d0d41a Merge 'Don't panic if doing INSERT INTO ... SELECT rowid' from Jussi Saurio
Backport: 0.2
Closes #3567

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3572
2025-10-04 10:11:09 +03:00
Pekka Enberg
b81d45ae11 Merge 'remove dyn DatabaseStorage replace it with DatabaseFile' from Pedro Muniz
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3566
2025-10-04 10:08:33 +03:00
Pere Diaz Bou
e3f2bbf90b core/mvcc: remove unnecessary fields in CommitState::Checkpoint 2025-10-03 22:27:03 +02:00
Pere Diaz Bou
c612a51df7 fmt 2025-10-03 22:24:38 +02:00
Pere Diaz Bou
6c1983aadf core/vdbe: op_checkpoint set update_transaction_state logical log checkpoint to true 2025-10-03 22:24:38 +02:00
Pere Diaz Bou
8e80af7221 core/mvcc: checkpoint if needed after commit 2025-10-03 22:24:38 +02:00
Pere Diaz Bou
a30c95d988 core/mvcc/logical-log: add threshold for checkpoints in logical log 2025-10-03 22:24:33 +02:00
Jussi Saurio
81b437c690 Don't panic if doing INSERT INTO ... SELECT rowid
Closes #3567
2025-10-03 23:12:24 +03:00
Jussi Saurio
8dac1ba21a Fix: actually enforce uniqueness in CREATE UNIQUE INDEX
...we just didn't do it
2025-10-03 22:58:42 +03:00
Jussi Saurio
cb96c3e944 VDBE: implement SorterCompare
this is used for uniqueness enforcement in CREATE INDEX.
2025-10-03 22:58:42 +03:00
pedrocarlo
f3dc0bef5d remove some explicit Arc<dyn File> references 2025-10-03 16:39:57 -03:00
pedrocarlo
e93add6c80 remove dyn DatabaseStorage and replace it with DatabaseFile 2025-10-03 14:14:15 -03:00
FHaggs
af35f6534d Fix: Calculate right buff len in the case of the first buffer in
try_pwritev_raw.
2025-10-03 18:46:16 +02:00
Pere Diaz Bou
55f5cd2ad7 core/mvcc/logical-log: test automatic checkpoint 2025-10-03 17:23:59 +02:00
Pere Diaz Bou
7db5a09005 core/mvcc/logical-log: checkpoint support not updating transaction_state if needed 2025-10-03 17:23:35 +02:00
Pere Diaz Bou
9c9d4d147e core/btree: fuzz tests force page 1 allocation with a transaction 2025-10-03 13:28:28 +02:00
Pere Diaz Bou
8f103f7c35 core/wal: introduce transaction_count, same as iChange in sqlite 2025-10-03 13:02:47 +02:00
Pekka Enberg
c98bf9b593 Merge 'core/wal: check index header on begin_write_tx' from Pere Diaz Bou
Fixes a page cache staleness issue where connections could incorrectly
believe the database hasn't changed after checkpointing. This can happen
when writes following a checkpoint resulted in the same `max_frame
value`, causing connections to miss updates since they only checked
`max_frame` to detect changes.

Closes #3502
2025-10-03 13:51:22 +03:00
Pere Diaz Bou
b5a969933c core/wal: remove dbg! 2025-10-03 12:17:35 +02:00
Jussi Saurio
ec6731de0a Disallow unexpected interop between WAL mode and MVCC mode
1. DB cannot be opened with MVCC if non-zero WAL file exists
2. DB cannot be opened without MVCC if non-zero logical log file exists
2025-10-03 12:00:35 +03:00
Pekka Enberg
d9e607082f Merge 'Disallow INDEXED BY / NOT INDEXED in select' from Jussi Saurio
Closes #3547

Closes #3551
2025-10-03 11:29:13 +03:00
Pekka Enberg
297aaf4887 core/mvcc: Rename "-lg" to "-log"
The "lg" name is just weird.
2025-10-03 10:08:02 +03:00
Jussi Saurio
b142dd42b5 Disallow INDEXED BY in select 2025-10-03 09:46:23 +03:00
Jussi Saurio
d2f5e67b25 Merge 'Fix COLLATE' from Jussi Saurio
Fixes the following problems with COLLATE:
- Fix: incorrectly used e.g. `x COLLATE NOCASE = 'fOo'` as index
constraint on an index whose column was not case-insensitively collated
- Fix: various ephemeral indexes (in GROUP BY, ORDER BY, DISTINCT) and
subqueries did not retain proper collation information of columns
- Fix: collation of a given expression was not determined properly
according to SQLite's rules
Adds TCL tests and fuzz test
Closes #3476
Closes #1524
Closes #3305

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3538
2025-10-03 09:34:24 +03:00
Pekka Enberg
33e727ce8f Merge 'core/mvcc: Return completions from logical log methods' from Pedro Muniz
`IOResult` implies we have a state machine that needs to be polled to
`Completion`, which is not the case here. We are just emitting the IO
operation in this case. This led us to never reaching the
`IOResult::Done` branch that actually fsynced the logical log in
`Checkpoint`.
I also sprinkled some
```rust
if c.is_completed() {
   Ok(TransitionResult::Continue)
} else {
   Ok(TransitionResult::Io(IOCompletions::Single(c)))
}
```
just to be more efficient with sync IO, but it is not strictly necessary
here.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3549
2025-10-03 09:29:31 +03:00
pedrocarlo
131a5b8048 adjust logical log IO functions to return Completions and not IOResult 2025-10-03 01:44:41 -03:00
Preston Thorpe
990740ab73 Merge 'core/vdbe: Don't clear cursors in ProgramState::reset()' from Pekka Enberg
We don't need to clear the cursors explicitly because OpenRead and
OpenWrite will anyway replace them.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3526
2025-10-02 17:32:05 -04:00
Jussi Saurio
58ea9e4c3c clippy 2025-10-02 21:49:33 +03:00
Jussi Saurio
8e2e557da4 Collate: fix Insn::Compare to use collation seq of each compared column 2025-10-02 21:49:33 +03:00
Jussi Saurio
edd4651b97 Collate: add proper collation info for GROUP BY sorter columns 2025-10-02 21:49:33 +03:00
Jussi Saurio
f02757fe11 Collate: add proper collation to FROM-clause subquery result cols 2025-10-02 21:49:33 +03:00
Jussi Saurio
edfe0cb4fe Collate: prevent using an index if collation sequences don't match 2025-10-02 21:49:33 +03:00
Jussi Saurio
d42f3c7cbb Collate: compute collations properly for ORDER BY 2025-10-02 21:49:33 +03:00
Jussi Saurio
5a5f49933d Collate: add proper collation info to DISTINCT indexes 2025-10-02 21:49:33 +03:00
Jussi Saurio
f4ee0457b2 Collate: add proper collation info to compound select deduplication indexes 2025-10-02 21:49:33 +03:00
Jussi Saurio
e1fcd7b5e9 Collate: add get_collseq_from_expr()
Determines collation sequence to use for a given Expr
based on SQLite collation rules.
2025-10-02 21:49:33 +03:00
PThorpe92
43aba0ee95 Fix integer affinity for rowid expr type 2025-10-02 14:29:53 -04:00