Commit Graph

720 Commits

Author SHA1 Message Date
pedrocarlo
0779c23bbf fix merge conflicts 2025-08-04 12:32:34 -03:00
pedrocarlo
d2019e95f3 pass schema to epilogue for schema_version checking + do not Pragma Schema Version in open_with_flags to avoid infinite loop in reprepare. Just access the database header directly 2025-08-04 12:32:34 -03:00
pedrocarlo
c567636deb Adjust Transaction OpCode to accept schema cookie + check if cookie changed 2025-08-04 12:32:34 -03:00
Jussi Saurio
506bb5f67f Merge 'Direct schema mutation – add instruction' from Levy A.
Resolves #2378.
```
`ALTER TABLE _ RENAME TO _`/limbo_rename_table/
                        time:   [15.645 ms 15.741 ms 15.850 ms]
Found 12 outliers among 100 measurements (12.00%)
  8 (8.00%) high mild
  4 (4.00%) high severe
`ALTER TABLE _ RENAME TO _`/sqlite_rename_table/
                        time:   [34.728 ms 35.260 ms 35.955 ms]
Found 15 outliers among 100 measurements (15.00%)
  8 (8.00%) high mild
  7 (7.00%) high severe
  ```
<img width="1000" height="199" alt="image" src="https://github.com/user-
attachments/assets/ad943355-b57d-43d9-8a84-850461b8af41" />

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2399
2025-08-04 16:55:38 +03:00
Pere Diaz Bou
f26e442597 core/mvcc: fix new rowid
next rowid was being tracked globally for all tables and restarted to 0
every time database was opened
2025-08-04 12:31:17 +02:00
Levy A.
b9a3a93ef0 fix: clippy 2025-08-02 20:06:05 -03:00
Levy A.
b14a11a2fd fix: change name for schema btree + fix benchmark 2025-08-02 17:17:36 -03:00
Jussi Saurio
130e1f80ea fix/vdbe: call seek_to_last() only once in op_new_rowid 2025-08-02 14:18:58 +03:00
Pekka Enberg
2c05a3e787 Merge 'perf/vdbe: remove eager cloning in op_comparison' from Jussi Saurio
Shaves off about 100-200ms of runtime from TPC-H `19.sql`

Closes #2385
2025-08-02 10:01:47 +03:00
Jussi Saurio
be1456f7cb Merge 'use state machine for NoConflict opcode' from Mikaël Francoeur
This will save some work when yielding to IO. Previously, on every
invocation, if the record was a packed record, we parsed it and iterated
through the values to check for nulls. Now, the pre-seeking work is done
only once.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2394
2025-08-02 09:37:00 +03:00
Levy A.
1e177053cb feat: add RenameTable instruction
direct schema mutation, no reparsing
2025-08-01 21:11:25 -03:00
Mikaël Francoeur
81412b4a17 use state machine for NoConflict opcode 2025-08-01 17:29:57 -04:00
rajajisai
f6d43df46f Merge branch 'tursodatabase:main' into issue/2077 2025-08-01 15:20:36 -04:00
rajajisai
d09dd4170b Format code 2025-08-01 11:59:57 -07:00
rajajisai
30c059483e Parse value as float if it cannot be parsed as integer(when the value cannot fit in i64) 2025-08-01 10:49:40 -07:00
rajajisai
7e84148883 Fix integer overflow check in number parser 2025-08-01 10:10:02 -07:00
Jussi Saurio
d58d71ad1b perf/vdbe: remove eager cloning in op_comparison 2025-08-01 14:04:56 +03:00
Pere Diaz Bou
5ad7d10790 core/mvcc: fix use of rwlock 2025-08-01 10:38:41 +02:00
Pere Diaz Bou
c4318cac36 core/mvcc: fix tests 2025-08-01 10:38:41 +02:00
Pere Diaz Bou
49a00ff338 core/mvcc: load table's rowid on initialization
We need to load rowids into mvcc's store in order before doing any read
in case there are rows.

This has a performance penalty for now as expected because we should,
ideally, scan for row ids lazily instead.
2025-08-01 10:38:41 +02:00
Pere Diaz Bou
b399ddea1b core/mvcc: begin pager read txn on mvcc begin_txn 2025-08-01 10:38:41 +02:00
Jussi Saurio
e147494642 pager: make WAL optional again and remove DummyWAL 2025-08-01 10:14:35 +03:00
Jussi Saurio
8c6293ebb7 VDBE: use temporary on-disk file for OpenEphemeral 2025-08-01 10:14:01 +03:00
Preston Thorpe
bd9df6262f Merge 'IN queries' from Glauber Costa
Merge 'IN queries' from Glauber Costa

Implement IN queries.
It is currently as todo!(), but my main motivation is that scavenging
for EXPLAINs, that pattern, at least in simple queries like SELECT ...
IN (1,2,3) uses the AddImm instruction we just added.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2342
2025-07-31 10:00:18 -04:00
Glauber Costa
9e8ba5263b Implement the AddImm opcode
It is a simple opcode. The hard part was finding a sqlite statement
that uses it =)
2025-07-31 08:08:07 -05:00
Jussi Saurio
9e1fca2eba vdbe: disallow checkpointing in interactive tx 2025-07-31 13:16:33 +03:00
Jussi Saurio
f619556344 Merge 'Direct DatabaseHeader reads and writes – with_header and with_header_mut' from Levy A.
This PR introduces two methods to pager. Very much inspired by
`with_schema` and `with_schema_mut`. `Pager::with_header` and
`Pager::with_header_mut` will give to the closure a shared and unique
reference respectively that are transmuted references from the `PageRef`
buffer.
This PR also adds type-safe wrappers for `Version`, `PageSize`,
`CacheSize` and `TextEncoding`, as they have special in-memory
representations.
Writing the `DatabaseHeader` is just a single `memcpy` now.
```rs
pub fn write_database_header(&self, header: &DatabaseHeader) {
    let buf = self.as_ptr();
    buf[0..DatabaseHeader::SIZE].copy_from_slice(bytemuck::bytes_of(header));
}
```
`HeaderRef` and `HeaderRefMut` are used in the `with_header*` methods,
but also can be used on its own when there are multiple reads and writes
to the header, where putting everything in a closure would add too much
nesting.

Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #2234
2025-07-31 10:02:47 +03:00
PThorpe92
07137c7aaf Merge 'Implement the Cast opcode' from Glauber Costa
Our compat matrix mentions a couple of opcodes: ToInt, ToBlob, etc.
Those opcodes do not exist.
Instead, there is a single Cast opcode, that takes the affinity as a
parameter.
Currently we just call a function when we need to cast. This PR fixes
the compat file, implements the cast opcode, and in at least one
instance, when explicitly using the CAST keyword, uses that opcode
instead of a function in the generated bytecode.

Reviewed-by: Preston Thorpe (@PThorpe92)
Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #2352
2025-07-30 22:32:09 -04:00
Glauber Costa
4bd1582e7d Implement the Cast opcode
Our compat matrix mentions a couple of opcodes: ToInt, ToBlob, etc.
Those opcodes do not exist.

Instead, there is a single Cast opcode, that takes the affinity as a
parameter.

Currently we just call a function when we need to cast. This PR fixes
the compat file, implements the cast opcode, and in at least one
instance, when explicitly using the CAST keyword, uses that opcode
instead of a function in the generated bytecode.
2025-07-30 20:44:54 -05:00
Levy A.
e35fdb8263 feat: zero-copy DatabaseHeader 2025-07-30 17:33:59 -03:00
Pekka Enberg
895f2acbfb Merge 'Fix concat_ws to match sqlite behavior' from bit-aloo
closes: #2101
Refactors exec_concat_ws to skip null and blob arguments instead of
inserting separators for them. Also adds a fuzz test.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2338
2025-07-30 21:31:58 +03:00
Jussi Saurio
7caef278a5 Merge 'Rewrite the WAL' from Preston Thorpe
closes #1893
Adds some fairly extensive tests but I'll continue to add some python
tests on top of the unit tests.
## Restart:
tested 
- open new DB
- create table and do a bunch of inserts
- `pragma wal_checkpoint(RESTART);`
- close db file
- re-open and verify we can read the wal/repopulate the frame cache
- verify min|max frame
tested 
- open same DB
- add more inserts
- `pragma wal_checkpoint(RESTART);`
- do _more_ inserts
- close
- re-open
- verify checksums/max_frame are valid
- verify row count
## Truncate
tested 
- open new db
- create table and add inserts
- `pragma wal_checkpoint(truncate);`
- close file
- verify WAL file is empty (32 bytes, header only)
- re-open file
- verify content/row count
tested 
- open db
- create table and insert many rows
- `pragma wal_checkpoint(truncate);`
- insert _more_ rows
- close db file
- verify WAL file is valid
- re-open file
- verify we can read entire file/repopulate the frame cache
<img width="541" height="315" alt="image" src="https://github.com/user-
attachments/assets/0470c795-5116-4866-b913-78c07b06b68c" />
```
# header
magic=0x377f0682
version=3007000
page_size=4096
seq=2
salt=ec475ff2-7ea94342
checksum=c9464aff-c571cc22
```

Closes #2179
2025-07-30 18:50:49 +03:00
Jussi Saurio
7b1f04dc5e pager: only ROLLBACK your own transaction, not if someone else is writing 2025-07-30 17:00:38 +03:00
Pekka Enberg
951bebfac3 Merge 'Add vector_concat and vector_slice support' from bit-aloo
Closes: #2323
This PR adds support for two new vector functions:
* vector_concat(x, y) – Concatenates two vectors of the same type.
* vector_slice(x, start_index, end_index) – Extracts a subvector from
the input vector.
Notes:
* Negative start_index or end_index is not supported

Reviewed-by: Nikita Sivukhin (@sivukhin)

Closes #2336
2025-07-30 16:58:38 +03:00
Jussi Saurio
b1aa13375d call pager.end_tx() everywhere instead of pager.rollback() 2025-07-30 16:39:38 +03:00
Jussi Saurio
fd5e73f038 op_transaction: read tx must be ended in all cases if begin_write_tx fails 2025-07-30 14:58:03 +03:00
Jussi Saurio
9a63425b43 clippy 2025-07-30 14:58:03 +03:00
bit-aloo
1d83bb48c5 fix exec_concat_ws implementation 2025-07-30 17:27:15 +05:30
PThorpe92
4dc15492d8 Integrate changes from tx isolation commits from @jussisaurio 2025-07-30 14:10:12 +03:00
PThorpe92
2c3a9fe5ef Finish wal transaction handling and add more wal and chkpt testing 2025-07-30 14:10:10 +03:00
PThorpe92
eaa6f99fa8 Hold and ensure release of proper locks if we trunc the db file post-checkpoint 2025-07-30 14:08:33 +03:00
PThorpe92
7643ef97a6 Pass checkpoint mode from sqlite3 c api argument 2025-07-30 14:08:33 +03:00
bit-aloo
96a99ca48a rename subvector to vector_slice 2025-07-30 13:34:49 +05:30
bit-aloo
a5dce2b50b add subvector execution flow 2025-07-30 09:51:08 +05:30
bit-aloo
e4d79a6516 add vec_concat execution flow 2025-07-30 06:07:03 +05:30
Diego Reis
e0b099f5ad refactor: Implement conversion between InsnFunctionStepResult and
StepResult
2025-07-29 15:02:09 -03:00
Pekka Enberg
8adc807cd7 Merge 'Change function signatures to return IO Completions' from Pedro Muniz
Changes a couple of function signatures to return `Completion`. Also, I
changed `Completion` to be internally `Arc` to abstract the `Arc`
implementation detail, and to be able to attach a `#[must_use]` to the
`Completion` struct, so that cargo check can show us where we are not
tracking completions in the code. I also attached a `#[must_use]` to
`IOResult` so that we can see the places that we are not propagating or
waiting for I/O, demonstrating locations where functions should be
reentrant and are not.
Also, while we are with this refactor in progress I want to relax the
Clippy CI lint on unused_variables.

Closes #2309
2025-07-29 12:41:14 +03:00
pedrocarlo
3831e0db39 convert must_use compile warnings to unused_variables to track locations where we need to refactor in the future 2025-07-28 16:09:26 -03:00
Diego Reis
bab10909c3 Disable extension loading for wasm
We should enable it later when wasm become more mature
2025-07-28 14:49:07 -03:00
Jussi Saurio
111c0032ae Always extend texts and blobs 2025-07-28 11:06:16 +03:00