Commit Graph

3757 Commits

Author SHA1 Message Date
PThorpe92
07137c7aaf Merge 'Implement the Cast opcode' from Glauber Costa
Our compat matrix mentions a couple of opcodes: ToInt, ToBlob, etc.
Those opcodes do not exist.
Instead, there is a single Cast opcode, that takes the affinity as a
parameter.
Currently we just call a function when we need to cast. This PR fixes
the compat file, implements the cast opcode, and in at least one
instance, when explicitly using the CAST keyword, uses that opcode
instead of a function in the generated bytecode.

Reviewed-by: Preston Thorpe (@PThorpe92)
Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #2352
2025-07-30 22:32:09 -04:00
Glauber Costa
4bd1582e7d Implement the Cast opcode
Our compat matrix mentions a couple of opcodes: ToInt, ToBlob, etc.
Those opcodes do not exist.

Instead, there is a single Cast opcode, that takes the affinity as a
parameter.

Currently we just call a function when we need to cast. This PR fixes
the compat file, implements the cast opcode, and in at least one
instance, when explicitly using the CAST keyword, uses that opcode
instead of a function in the generated bytecode.
2025-07-30 20:44:54 -05:00
Jussi Saurio
e128bd477e Merge 'Support VALUES clauses for compound select' from meteorgan
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2293
2025-07-30 21:34:40 +03:00
Jussi Saurio
bc148a3906 Merge 'skip invalid inputs in cosine distance prop test' from bit-aloo
closes: #1790
Updates test_vector_distance to treat invalid inputs as non-errors,
skipping them instead. These cases aren't considered real errors, so no
explicit error handling is needed in the test.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2341
2025-07-30 21:33:15 +03:00
Pekka Enberg
895f2acbfb Merge 'Fix concat_ws to match sqlite behavior' from bit-aloo
closes: #2101
Refactors exec_concat_ws to skip null and blob arguments instead of
inserting separators for them. Also adds a fuzz test.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2338
2025-07-30 21:31:58 +03:00
Pekka Enberg
147f105136 Merge 'core/mvcc: Switch to parking_lot RwLock' from Pekka Enberg
Closes #2344
2025-07-30 21:31:31 +03:00
Pekka Enberg
b7cb4a3ed4 core/mvcc: Switch to parking_lot RwLock 2025-07-30 20:25:45 +03:00
bit-aloo
1be93c8c18 skip invalid inputs in cosine distance prop test 2025-07-30 21:27:55 +05:30
Jussi Saurio
7caef278a5 Merge 'Rewrite the WAL' from Preston Thorpe
closes #1893
Adds some fairly extensive tests but I'll continue to add some python
tests on top of the unit tests.
## Restart:
tested 
- open new DB
- create table and do a bunch of inserts
- `pragma wal_checkpoint(RESTART);`
- close db file
- re-open and verify we can read the wal/repopulate the frame cache
- verify min|max frame
tested 
- open same DB
- add more inserts
- `pragma wal_checkpoint(RESTART);`
- do _more_ inserts
- close
- re-open
- verify checksums/max_frame are valid
- verify row count
## Truncate
tested 
- open new db
- create table and add inserts
- `pragma wal_checkpoint(truncate);`
- close file
- verify WAL file is empty (32 bytes, header only)
- re-open file
- verify content/row count
tested 
- open db
- create table and insert many rows
- `pragma wal_checkpoint(truncate);`
- insert _more_ rows
- close db file
- verify WAL file is valid
- re-open file
- verify we can read entire file/repopulate the frame cache
<img width="541" height="315" alt="image" src="https://github.com/user-
attachments/assets/0470c795-5116-4866-b913-78c07b06b68c" />
```
# header
magic=0x377f0682
version=3007000
page_size=4096
seq=2
salt=ec475ff2-7ea94342
checksum=c9464aff-c571cc22
```

Closes #2179
2025-07-30 18:50:49 +03:00
Jussi Saurio
ff1c1b6b8c wal_insert_end: call pager.rollback() after tx ends so that lock index is preserved when ending tx 2025-07-30 18:22:40 +03:00
Jussi Saurio
7240d7903c fmt 2025-07-30 18:22:17 +03:00
Jussi Saurio
7bc11fe2f9 wal_insert_end: revert unintentional changes 2025-07-30 18:16:23 +03:00
Jussi Saurio
2813a7a5de clippy 2025-07-30 17:25:30 +03:00
Jussi Saurio
c00d1fcfc0 fmt 2025-07-30 17:21:29 +03:00
Jussi Saurio
66c4b44c55 pager: call rollback() after ending txn so that read lock info is not lost when ending txn 2025-07-30 17:21:19 +03:00
Jussi Saurio
7b1f04dc5e pager: only ROLLBACK your own transaction, not if someone else is writing 2025-07-30 17:00:38 +03:00
Pekka Enberg
951bebfac3 Merge 'Add vector_concat and vector_slice support' from bit-aloo
Closes: #2323
This PR adds support for two new vector functions:
* vector_concat(x, y) – Concatenates two vectors of the same type.
* vector_slice(x, start_index, end_index) – Extracts a subvector from
the input vector.
Notes:
* Negative start_index or end_index is not supported

Reviewed-by: Nikita Sivukhin (@sivukhin)

Closes #2336
2025-07-30 16:58:38 +03:00
PThorpe92
e7eda25802 Make sure to end read tx on error of wal insert begin API 2025-07-30 09:44:29 -04:00
Jussi Saurio
b1aa13375d call pager.end_tx() everywhere instead of pager.rollback() 2025-07-30 16:39:38 +03:00
Jussi Saurio
975b7b5434 wal: fix test incorrect expectation 2025-07-30 15:53:13 +03:00
Jussi Saurio
af660326d8 finish_append_frames_commit: revert bumping readmark incorrectly 2025-07-30 15:53:01 +03:00
Jussi Saurio
43d1321033 ignore completion result of self.read_frame 2025-07-30 14:58:03 +03:00
Jussi Saurio
338cab3f28 End read transaction when Schema::make_from_btree fails 2025-07-30 14:58:03 +03:00
Jussi Saurio
fd5e73f038 op_transaction: read tx must be ended in all cases if begin_write_tx fails 2025-07-30 14:58:03 +03:00
Jussi Saurio
9a63425b43 clippy 2025-07-30 14:58:03 +03:00
Jussi Saurio
772b71963e finish_append_frames_commit: properly increase readmark on commit 2025-07-30 14:58:03 +03:00
Jussi Saurio
1562c1df10 begin_read_tx: better assertion failure message 2025-07-30 14:58:03 +03:00
bit-aloo
1d83bb48c5 fix exec_concat_ws implementation 2025-07-30 17:27:15 +05:30
PThorpe92
4dc15492d8 Integrate changes from tx isolation commits from @jussisaurio 2025-07-30 14:10:12 +03:00
PThorpe92
2c3a9fe5ef Finish wal transaction handling and add more wal and chkpt testing 2025-07-30 14:10:10 +03:00
PThorpe92
8806b77d26 Clear snapshot and readmark/lock index flags on failure 2025-07-30 14:09:18 +03:00
PThorpe92
d702e6a80c Polish checkpointing and fix tests, add documentation 2025-07-30 14:08:53 +03:00
PThorpe92
8ec99a9143 Remove assert for !NO_LOCK_HELD, properly handle writing header if reset 2025-07-30 14:08:51 +03:00
PThorpe92
529cc14e29 Fix wal tests remove unwrap from previous Result return val 2025-07-30 14:08:33 +03:00
PThorpe92
7640535ba4 Fix transaction read0 shortcut in WAL and track whether we have snapshot 2025-07-30 14:08:33 +03:00
PThorpe92
ff1987a45c Temporarily remove optimization for new read tx to grab read mark 0 and skip db file 2025-07-30 14:08:33 +03:00
PThorpe92
318bfa9590 Change incorrect comments and rename guard 2025-07-30 14:08:33 +03:00
PThorpe92
1490a586b1 Apply suggestions/fixes and add extensive comments to wal chkpt 2025-07-30 14:08:33 +03:00
PThorpe92
3e75444388 Remove panic in cacheflush io.block in pager now that checkpoitns can return busy 2025-07-30 14:08:33 +03:00
PThorpe92
3db72cf111 Just forget Full checkpoint mode for now, comment out compat test 2025-07-30 14:08:33 +03:00
PThorpe92
49f90980d4 Create new header after truncation chkpt 2025-07-30 14:08:33 +03:00
PThorpe92
b214c3dfc8 Add diff chkpt modes to sqlite3 api, finish checkpoint logic and add tests 2025-07-30 14:08:33 +03:00
PThorpe92
eaa6f99fa8 Hold and ensure release of proper locks if we trunc the db file post-checkpoint 2025-07-30 14:08:33 +03:00
PThorpe92
8ca37b71b6 Ensure we properly hold and release read locks in log restart method and fix tests 2025-07-30 14:08:33 +03:00
PThorpe92
9b7e5ed292 Trunc db file after backfilling everything in calling method 2025-07-30 14:08:33 +03:00
PThorpe92
7643ef97a6 Pass checkpoint mode from sqlite3 c api argument 2025-07-30 14:08:33 +03:00
PThorpe92
f4becd1296 Allow using !passive checkpoint methods in pragma wal_checkpoint 2025-07-30 14:08:33 +03:00
PThorpe92
1a9b7ef76e Add support for truncate, restart and full checkpointing methods 2025-07-30 14:08:31 +03:00
PThorpe92
6b71efa5b6 Accept mode argument for checkpoint in connection api 2025-07-30 14:07:20 +03:00
PThorpe92
441319e774 Fix todo! in vfs IO preventing write wal header from using wait_for_completion 2025-07-30 14:07:19 +03:00