Commit Graph

349 Commits

Author SHA1 Message Date
Pekka Enberg
f8bdc02986 Merge 'Rename Completion methods' from Pedro Muniz
Reviewed-by: bit-aloo (@Shourya742)
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3579
2025-10-06 20:12:21 +03:00
Pere Diaz Bou
59c08c1062 Merge 'core/mvcc: implement PartialOrd for RowId' from Pere Diaz Bou
Closes #3597
2025-10-06 16:42:04 +02:00
pedrocarlo
5a7390735d rename Completion functions 2025-10-06 11:07:06 -03:00
Pekka Enberg
41d909e1fb Merge 'MVCC: do checkpoint writes in ascending order of rowid' from Jussi Saurio
For insert-heavy checkpoints this gives a much higher chance of using
the balance-quick subalgorithm instead of the more complex and slower
balance-nonroot.

Closes #3589
2025-10-06 16:30:19 +03:00
Pere Diaz Bou
aed255d2e6 core/mvcc: implement PartialOrd for RowId 2025-10-06 13:43:42 +02:00
Pere Diaz Bou
fc7e1639a0 core/mvcc: filter out seek results where is not same table_id 2025-10-06 13:29:10 +02:00
Pere Diaz Bou
b9b9831d17 core/mvcc: test seek with empty table 2025-10-06 13:28:51 +02:00
Jussi Saurio
fd9008f123 MVCC: do checkpoint writes in ascending order of rowid
For insert-heavy checkpoints this gives a much higher chance of using
the balance-quick subalgorithm instead of the more complex and slower
balance-nonroot.
2025-10-06 10:11:19 +03:00
Pere Diaz Bou
e3f2bbf90b core/mvcc: remove unnecessary fields in CommitState::Checkpoint 2025-10-03 22:27:03 +02:00
Pere Diaz Bou
c612a51df7 fmt 2025-10-03 22:24:38 +02:00
Pere Diaz Bou
8e80af7221 core/mvcc: checkpoint if needed after commit 2025-10-03 22:24:38 +02:00
Pere Diaz Bou
a30c95d988 core/mvcc/logical-log: add threshold for checkpoints in logical log 2025-10-03 22:24:33 +02:00
Pere Diaz Bou
55f5cd2ad7 core/mvcc/logical-log: test automatic checkpoint 2025-10-03 17:23:59 +02:00
Pere Diaz Bou
7db5a09005 core/mvcc/logical-log: checkpoint support not updating transaction_state if needed 2025-10-03 17:23:35 +02:00
Pekka Enberg
297aaf4887 core/mvcc: Rename "-lg" to "-log"
The "lg" name is just weird.
2025-10-03 10:08:02 +03:00
pedrocarlo
131a5b8048 adjust logical log IO functions to return Completions and not IOResult 2025-10-03 01:44:41 -03:00
Jussi Saurio
9e4ea6ea34 Merge 'core/mvcc/logical-log: fail in read_more_data if couldn't read enough' from Pere Diaz Bou
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3500
2025-10-02 06:58:28 +03:00
Jussi Saurio
bb4e54ca73 Merge 'fix/mvcc: deserialize table_id as i64' from Jussi Saurio
Reviewed-by: Nikita Sivukhin (@sivukhin)

Closes #3492
2025-10-02 06:58:01 +03:00
Pere Diaz Bou
b89df44339 fmt 2025-10-01 14:31:30 +02:00
Pere Diaz Bou
dc0245d758 core/mvcc/logical-log: fail in read_more_data if couldn't read enough 2025-10-01 14:25:22 +02:00
Jussi Saurio
e9f0c59bcc fix/mvcc: set log offset to end of file after recovery finishes
otherwise we start overwriting existing log entries
2025-10-01 12:46:24 +03:00
Jussi Saurio
bcb941f33b fix/mvcc: deserialize table_id as i64 2025-10-01 10:26:23 +03:00
Jussi Saurio
d4d50b564a fix even more tests 2025-09-30 23:22:07 +03:00
Jussi Saurio
adc5b7b27f remove monkey print 2025-09-30 22:57:21 +03:00
Jussi Saurio
fe871188bf fix tests again 2025-09-30 22:54:48 +03:00
Jussi Saurio
fb2878973f fix sort order of write set 2025-09-30 22:54:36 +03:00
Jussi Saurio
fd84fd0683 fix test compilation errors 2025-09-30 22:27:28 +03:00
Jussi Saurio
e68c652f8f Add some table ID integrity checks to logical log recovery 2025-09-30 22:27:28 +03:00
Jussi Saurio
64ce33bd5c Move resolution of tableid/rootpage inside MvCursor constructor 2025-09-30 17:04:37 +03:00
Jussi Saurio
7c897d382f Implement MvTableId newtype for better type safety of table ids 2025-09-30 16:54:22 +03:00
Jussi Saurio
0ba4c6c00e use negative table id in mvcc tests 2025-09-30 16:53:12 +03:00
Jussi Saurio
a52dbb7842 Handle table ID / rootpages properly for both checkpointed and non-checkpointed tables
Table ID is an opaque identifier that is only meaningful to the MV store.
Each checkpointed MVCC table corresponds to a single B-tree on the pager,
which naturally has a root page.

We cannot use root page as the MVCC table ID directly because:
- We assign table IDs during MVCC commit, but
- we commit pages to the pager only during checkpoint
which means the root page is not easily knowable ahead of time.

Hence, we:

- store the mapping between table id and btree rootpage
- sqlite_schema rows will have a negative rootpage column if the
  table has not been checkpointed yet.
2025-09-30 16:53:12 +03:00
Pekka Enberg
9788f6d005 Merge 'core/mvcc: Optimize exclusive transaction check' from Pekka Enberg
The check is in fastpath so switch to an atomic instead.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3458
2025-09-30 16:02:42 +03:00
Pekka Enberg
3d327ba63c core/mvcc: Optimize exclusive transaction check
The check is in fastpath so switch to an atomic instead.
2025-09-30 15:00:24 +03:00
Pere Diaz Bou
0f631101df core: change page idx type from usize to i64
MVCC is like the annoying younger cousin (I know because I was him) that
needs to be treated differently. MVCC requires us to use root_pages that
might not be allocated yet, and the plan is to use negative root_pages
for that case. Therefore, we need i64 in order to fit this change.
2025-09-29 18:38:43 +02:00
Pekka Enberg
65b382b9e1 Merge 'Make MVCC code Send and Sync' from Pekka Enberg
Closes #3361
2025-09-27 08:21:01 +03:00
Pekka Enberg
a932ac5450 Merge 'core: recover logical log on Database::connect ' from Pere Diaz Bou
If we open database and logical log is not empty we need to recover from
it. We also make sure a single recover executes concurrently and other
connections just wait for it to finish.
I also changed the fuzz tester to use `restart` instead of calling
`load_logical_log` manually to test this behaviour.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3359
2025-09-27 08:20:41 +03:00
Pere Diaz Bou
99adf73168 core/mvcc/logical-log: rename to needs_recovery 2025-09-26 16:59:57 +02:00
Pere Diaz Bou
9e47cc3700 clippy 2025-09-26 14:16:11 +02:00
Pekka Enberg
1402e9841e core/mvcc: Wrap StreamingLogicalLogReader::buffer with RwLock 2025-09-26 14:19:16 +03:00
Pekka Enberg
96accef06c core/mvcc: Wrap header with RwLock 2025-09-26 14:10:18 +03:00
Pere Diaz Bou
9c1d94a355 core/mvcc/logical-log: assert we don't call begin_load_tx twice 2025-09-26 12:59:13 +02:00
Pere Diaz Bou
4cdf293a2b core/mvcc/logical-log: fuzz test recover use db.restart 2025-09-26 12:56:58 +02:00
Pere Diaz Bou
ae994146af core/mvcc/logical-log: on mvcc restart clear DATABASE_MANAGER 2025-09-26 12:56:43 +02:00
Pere Diaz Bou
83d8a7c775 core/mvcc/logical-log: lock recover logical log process 2025-09-26 12:47:31 +02:00
Pekka Enberg
27ee12b300 Merge 'core/mvcc/logical-log: refactor get log path in tests' from Pere Diaz Bou
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3346
2025-09-25 21:04:15 +03:00
Pekka Enberg
9c9b525f8a Merge 'core: Wrap Connection::mv_tx with RwLock' from Pekka Enberg
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #3332
2025-09-25 20:47:03 +03:00
Pere Diaz Bou
334da8abbb core/mvcc/logical-log: refactor get log path in tests 2025-09-25 18:33:26 +02:00
Jussi Saurio
c07a227560 Merge 'core/mvcc/logical-log: load logical log from disk' from Pere Diaz Bou
## Description
Read transactions from logical log and load mvcc store based on the
contents onto transaction 0, which is the special transaction where we
will load all row versions that can be read by all new transactions.
## Todo
- [x] Testing for multiple transactions
- [ ] Lock multiple loading of same logical-log at the same time
- [x] Add column_count to format because if not it is hard to judge from
record alone.
- [x] Trim buffer read
I was thinking to implement the rest on another pr to not increase load
to review this one.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3323
2025-09-25 16:26:34 +03:00
Pere Diaz Bou
8f3332377a core/mvcc/logical-log: fmt fmt fmt fmt 2025-09-25 13:33:16 +02:00