Commit Graph

3760 Commits

Author SHA1 Message Date
PThorpe92
0f94cdef03 Fix io_uring pwritev to properly handle partial writes 2025-07-30 19:42:50 -04:00
PThorpe92
88445328a5 Handle partial writes for pwritev calls in io_uring and fix JS bindings 2025-07-30 19:42:50 -04:00
PThorpe92
5f01eaae35 Fix default io:;File::pwritev impl 2025-07-30 19:42:49 -04:00
PThorpe92
62f004c898 Fix write counter for writev batching in checkpoint 2025-07-30 19:42:49 -04:00
PThorpe92
7b2163208b batch backfilling pages when checkpointing 2025-07-30 19:42:48 -04:00
Jussi Saurio
e128bd477e Merge 'Support VALUES clauses for compound select' from meteorgan
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2293
2025-07-30 21:34:40 +03:00
Jussi Saurio
bc148a3906 Merge 'skip invalid inputs in cosine distance prop test' from bit-aloo
closes: #1790
Updates test_vector_distance to treat invalid inputs as non-errors,
skipping them instead. These cases aren't considered real errors, so no
explicit error handling is needed in the test.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2341
2025-07-30 21:33:15 +03:00
Pekka Enberg
895f2acbfb Merge 'Fix concat_ws to match sqlite behavior' from bit-aloo
closes: #2101
Refactors exec_concat_ws to skip null and blob arguments instead of
inserting separators for them. Also adds a fuzz test.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2338
2025-07-30 21:31:58 +03:00
Pekka Enberg
147f105136 Merge 'core/mvcc: Switch to parking_lot RwLock' from Pekka Enberg
Closes #2344
2025-07-30 21:31:31 +03:00
Pekka Enberg
b7cb4a3ed4 core/mvcc: Switch to parking_lot RwLock 2025-07-30 20:25:45 +03:00
bit-aloo
1be93c8c18 skip invalid inputs in cosine distance prop test 2025-07-30 21:27:55 +05:30
Jussi Saurio
7caef278a5 Merge 'Rewrite the WAL' from Preston Thorpe
closes #1893
Adds some fairly extensive tests but I'll continue to add some python
tests on top of the unit tests.
## Restart:
tested 
- open new DB
- create table and do a bunch of inserts
- `pragma wal_checkpoint(RESTART);`
- close db file
- re-open and verify we can read the wal/repopulate the frame cache
- verify min|max frame
tested 
- open same DB
- add more inserts
- `pragma wal_checkpoint(RESTART);`
- do _more_ inserts
- close
- re-open
- verify checksums/max_frame are valid
- verify row count
## Truncate
tested 
- open new db
- create table and add inserts
- `pragma wal_checkpoint(truncate);`
- close file
- verify WAL file is empty (32 bytes, header only)
- re-open file
- verify content/row count
tested 
- open db
- create table and insert many rows
- `pragma wal_checkpoint(truncate);`
- insert _more_ rows
- close db file
- verify WAL file is valid
- re-open file
- verify we can read entire file/repopulate the frame cache
<img width="541" height="315" alt="image" src="https://github.com/user-
attachments/assets/0470c795-5116-4866-b913-78c07b06b68c" />
```
# header
magic=0x377f0682
version=3007000
page_size=4096
seq=2
salt=ec475ff2-7ea94342
checksum=c9464aff-c571cc22
```

Closes #2179
2025-07-30 18:50:49 +03:00
Jussi Saurio
ff1c1b6b8c wal_insert_end: call pager.rollback() after tx ends so that lock index is preserved when ending tx 2025-07-30 18:22:40 +03:00
Jussi Saurio
7240d7903c fmt 2025-07-30 18:22:17 +03:00
Jussi Saurio
7bc11fe2f9 wal_insert_end: revert unintentional changes 2025-07-30 18:16:23 +03:00
Jussi Saurio
2813a7a5de clippy 2025-07-30 17:25:30 +03:00
Jussi Saurio
c00d1fcfc0 fmt 2025-07-30 17:21:29 +03:00
Jussi Saurio
66c4b44c55 pager: call rollback() after ending txn so that read lock info is not lost when ending txn 2025-07-30 17:21:19 +03:00
Jussi Saurio
7b1f04dc5e pager: only ROLLBACK your own transaction, not if someone else is writing 2025-07-30 17:00:38 +03:00
Pekka Enberg
951bebfac3 Merge 'Add vector_concat and vector_slice support' from bit-aloo
Closes: #2323
This PR adds support for two new vector functions:
* vector_concat(x, y) – Concatenates two vectors of the same type.
* vector_slice(x, start_index, end_index) – Extracts a subvector from
the input vector.
Notes:
* Negative start_index or end_index is not supported

Reviewed-by: Nikita Sivukhin (@sivukhin)

Closes #2336
2025-07-30 16:58:38 +03:00
PThorpe92
e7eda25802 Make sure to end read tx on error of wal insert begin API 2025-07-30 09:44:29 -04:00
Jussi Saurio
b1aa13375d call pager.end_tx() everywhere instead of pager.rollback() 2025-07-30 16:39:38 +03:00
Jussi Saurio
975b7b5434 wal: fix test incorrect expectation 2025-07-30 15:53:13 +03:00
Jussi Saurio
af660326d8 finish_append_frames_commit: revert bumping readmark incorrectly 2025-07-30 15:53:01 +03:00
Jussi Saurio
43d1321033 ignore completion result of self.read_frame 2025-07-30 14:58:03 +03:00
Jussi Saurio
338cab3f28 End read transaction when Schema::make_from_btree fails 2025-07-30 14:58:03 +03:00
Jussi Saurio
fd5e73f038 op_transaction: read tx must be ended in all cases if begin_write_tx fails 2025-07-30 14:58:03 +03:00
Jussi Saurio
9a63425b43 clippy 2025-07-30 14:58:03 +03:00
Jussi Saurio
772b71963e finish_append_frames_commit: properly increase readmark on commit 2025-07-30 14:58:03 +03:00
Jussi Saurio
1562c1df10 begin_read_tx: better assertion failure message 2025-07-30 14:58:03 +03:00
bit-aloo
1d83bb48c5 fix exec_concat_ws implementation 2025-07-30 17:27:15 +05:30
PThorpe92
4dc15492d8 Integrate changes from tx isolation commits from @jussisaurio 2025-07-30 14:10:12 +03:00
PThorpe92
2c3a9fe5ef Finish wal transaction handling and add more wal and chkpt testing 2025-07-30 14:10:10 +03:00
PThorpe92
8806b77d26 Clear snapshot and readmark/lock index flags on failure 2025-07-30 14:09:18 +03:00
PThorpe92
d702e6a80c Polish checkpointing and fix tests, add documentation 2025-07-30 14:08:53 +03:00
PThorpe92
8ec99a9143 Remove assert for !NO_LOCK_HELD, properly handle writing header if reset 2025-07-30 14:08:51 +03:00
PThorpe92
529cc14e29 Fix wal tests remove unwrap from previous Result return val 2025-07-30 14:08:33 +03:00
PThorpe92
7640535ba4 Fix transaction read0 shortcut in WAL and track whether we have snapshot 2025-07-30 14:08:33 +03:00
PThorpe92
ff1987a45c Temporarily remove optimization for new read tx to grab read mark 0 and skip db file 2025-07-30 14:08:33 +03:00
PThorpe92
318bfa9590 Change incorrect comments and rename guard 2025-07-30 14:08:33 +03:00
PThorpe92
1490a586b1 Apply suggestions/fixes and add extensive comments to wal chkpt 2025-07-30 14:08:33 +03:00
PThorpe92
3e75444388 Remove panic in cacheflush io.block in pager now that checkpoitns can return busy 2025-07-30 14:08:33 +03:00
PThorpe92
3db72cf111 Just forget Full checkpoint mode for now, comment out compat test 2025-07-30 14:08:33 +03:00
PThorpe92
49f90980d4 Create new header after truncation chkpt 2025-07-30 14:08:33 +03:00
PThorpe92
b214c3dfc8 Add diff chkpt modes to sqlite3 api, finish checkpoint logic and add tests 2025-07-30 14:08:33 +03:00
PThorpe92
eaa6f99fa8 Hold and ensure release of proper locks if we trunc the db file post-checkpoint 2025-07-30 14:08:33 +03:00
PThorpe92
8ca37b71b6 Ensure we properly hold and release read locks in log restart method and fix tests 2025-07-30 14:08:33 +03:00
PThorpe92
9b7e5ed292 Trunc db file after backfilling everything in calling method 2025-07-30 14:08:33 +03:00
PThorpe92
7643ef97a6 Pass checkpoint mode from sqlite3 c api argument 2025-07-30 14:08:33 +03:00
PThorpe92
f4becd1296 Allow using !passive checkpoint methods in pragma wal_checkpoint 2025-07-30 14:08:33 +03:00