Commit Graph

543 Commits

Author SHA1 Message Date
Jussi Saurio
5aa788691b Merge 'Fix math functions compatibility issues' from Levy A.
Adds `round`, `hex`, `unhex`, `abs`, `lower`, `upper`, `sign` and `log`
(with base) to the expression fuzzer.
Rounding with the precision argument still has some incompatibilities.

Closes #3160
2025-09-19 09:15:11 +03:00
Pere Diaz Bou
d53c64e84b core/schema: parse schema rows for MVCC transactions 2025-09-18 18:22:06 +02:00
Levy A.
85e0f1444d fix: add log with base to fuzzer 2025-09-18 13:09:39 -03:00
Levy A.
5016123214 fix: compatible rounding precision 2025-09-18 11:10:15 -03:00
Levy A.
2b632d4fe0 fix upper, lower and sign tests 2025-09-18 11:08:15 -03:00
Pekka Enberg
ecbe2c231f core: Rename Connection::_db to db 2025-09-18 16:00:41 +03:00
Pekka Enberg
2a5284afb9 core/storage: Use AtomicU32 for Pager::page_size 2025-09-18 11:33:32 +03:00
Nikita Sivukhin
ed819c9865 Merge branch 'main' into more-async 2025-09-18 10:48:54 +04:00
Pekka Enberg
d2376a239a Merge 'core/mvcc: introduce with_header for MVCC header update tracking' from Pere Diaz Bou
Currently header changes are tracked through pager by reading page 1.
MVCC has it's own layer to track changes during txn so this commit makes
it so that headers are tracked by each txn separately.
On commit we update the _global_ header which is used to update
`database_size` because pager commits require it to be up to date. This
also makes it _simpler_ to keep track of header updates and update
pager's header accordingly.
This PR is needed in order to make logical log work because we don't
want to rely on pager as much as possible!

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3156
2025-09-18 08:13:14 +03:00
Nikita Sivukhin
bf5397dade fix op_sorter 2025-09-17 21:30:40 +04:00
Nikita Sivukhin
dea041b7c0 fix after rebase 2025-09-17 19:06:07 +04:00
Nikita Sivukhin
a8bc06ea9c fix clippy 2025-09-17 19:06:07 +04:00
Nikita Sivukhin
c4c9d409c9 remove io.block from op_checkpoint 2025-09-17 19:06:07 +04:00
Nikita Sivukhin
27627bdb8d remove IOExt import from execute 2025-09-17 19:06:07 +04:00
Nikita Sivukhin
85b671997e remove io.block from op_open_ephemeral 2025-09-17 19:06:07 +04:00
Nikita Sivukhin
18feacf824 remove io.block from op_sorter_open 2025-09-17 19:06:07 +04:00
Nikita Sivukhin
e250285513 make op_transaction non-blocking 2025-09-17 19:06:07 +04:00
Pere Diaz Bou
88307197cc with_header mvcc update rc to arc 2025-09-17 11:50:24 +02:00
Pekka Enberg
182565fe0c core: Wrap MvCursor in Arc<RwLock<>>
Make it Send and Sync.
2025-09-17 12:46:55 +03:00
Pere Diaz Bou
64616dc2ca core/mvcc: introduce with_header for MVCC header update tracking
Currently header changes are tracked through pager by reading page 1.
MVCC has it's own layer to track changes during txn so this commit makes
it so that headers are tracked by each txn separately.

On commit we update the _global_ header which is used to update
`database_size` because pager commits require it to be up to date. This
also makes it _simpler_ to keep track of header updates and update
pager's header accordingly.
2025-09-17 11:42:44 +02:00
Jussi Saurio
9a2797963a Merge 'Remove LimboResult enum and InsnFunctionStepResult::Busy variant' from Jussi Saurio
We can just use `LimboError::Busy` for both of these.

Reviewed-by: Pekka Enberg <penberg@iki.fi>

Closes #3170
2025-09-17 12:06:54 +03:00
Jussi Saurio
e066e4f747 clippy 2025-09-17 11:25:50 +03:00
Jussi Saurio
b9ceacc356 Remove InsnFunctionStepResult::Busy
we don't need all these busy variants, let's just handle
LimboError::Busy
2025-09-17 11:22:49 +03:00
Jussi Saurio
dc103da2ed Remove LimboResult
this is only used for returning LimboResult::Busy, and we already
have LimboError::Busy, so it only adds confusion.

Moreover, the current busy handler was not handling LimboError::Busy,
because it's returned as an error, not as Ok. So this may fix the
"busy handler not working" issue in the perf thrpt benchmark.
2025-09-17 11:04:44 +03:00
Pekka Enberg
50653258cf core: Wrap symbol table with RwLock
Make it Send.
2025-09-17 10:58:32 +03:00
Pekka Enberg
17e9f05ea4 core: Convert Rc<Pager> to Arc<Pager> 2025-09-17 09:32:49 +03:00
Jussi Saurio
fad8d0c8b8 fix build 2025-09-17 08:45:13 +03:00
Jussi Saurio
cae234818b Merge 'Inital support for window functions' from Piotr Rżysko
This adds basic support for window functions. For now:
* Only existing aggregate functions can be used as window functions.
* Specialized window-specific functions (`rank`, `row_number`, etc.) are
not yet supported.
* Only the default frame definition is implemented:
`RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW EXCLUDE NO OTHERS`.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3079
2025-09-17 08:29:16 +03:00
Levy A.
fcc03d2f6a fix: compatible math functions
also add math functions to fuzzer
2025-09-17 00:23:27 -03:00
Jussi Saurio
d493a72cc0 dont unwrap begin_tx 2025-09-15 14:48:26 +03:00
Jussi Saurio
8f43741513 fix mvcc rollback
executing ROLLBACK did not rollback the mv-store transaction
2025-09-15 09:29:08 +03:00
Jussi Saurio
2ca1640a2a not always write 2025-09-14 22:24:07 +03:00
Jussi Saurio
396091044e store tx_mode in conn.mv_tx
otherwise op_transaction works completely wrong because each separate
insert statement overrides the tx_mode to Write
2025-09-14 21:59:08 +03:00
Jussi Saurio
7fe25a1d0e mvcc: remove conn.mv_transactions
afaict this isn't needed for anything since there is already
conn.mv_tx_id
2025-09-14 21:26:58 +03:00
Jussi Saurio
5feb9ea2f0 mvcc: fix non-concurrent transaction semantics
on the main branch, mvcc allows concurrent inserts from multiple
txns even without BEGIN CONCURRENT, and then always hangs whenever
one of the txns tries to commit.

this commit fixes that issue.
2025-09-14 21:23:06 +03:00
Piotr Rzysko
b911e80607 Add AggValue instruction
Adds the AggValue instruction, which computes the current aggregate
result and writes it to a dedicated destination register.

Unlike AggFinal, it does not overwrite or clear the accumulator
register. This makes it possible to retrieve aggregate results multiple
times—needed when processing window functions—while preserving the
accumulator state.
2025-09-13 10:49:14 +02:00
Piotr Rzysko
c5a12f52c2 Don't mutate state in op_agg_final
Previously, only the External and Avg aggregates mutated state during
AggFinal. This is unnecessary because AggFinal runs only once per group,
so caching the result provides no performance benefit.

By avoiding state mutation, we can also reuse op_agg_final for the
AggValue instruction that will be added soon.
2025-09-13 10:49:14 +02:00
Piotr Rzysko
867bef55d8 Add ResetSorter instruction
This instruction isn't used yet, but it will be needed for window
functions, since they heavily rely on ephemeral tables.
2025-09-13 10:44:56 +02:00
Piotr Rzysko
ea9599681e Add OpenDup instruction
The instruction isn’t used yet, but it’ll be needed for window functions,
since they heavily rely on ephemeral tables.
2025-09-13 10:35:33 +02:00
Pekka Enberg
ad6157028e Merge 'core/vdbe: Fix BEGIN CONCURRENT transactions' from Pekka Enberg
The transaction upgrade logic in Transaction opcode is total nonsense
for concurrent transactions so just drop it.
Fixes #3061

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #3070
2025-09-12 23:11:12 +03:00
Pekka Enberg
5e2b1bc0d3 Merge 'Fix incompatible math functions' from Levy A.
Fixes #1817, #2068, #1326, #1397.
The solution is very much not ideal, but fixes all math function related
incompatibilities.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3033
2025-09-12 21:28:08 +03:00
Pekka Enberg
86dcdad3d0 core/vdbe: Fix BEGIN CONCURRENT transactions
The transaction upgrade logic in Transaction opcode is total nonsense
for concurrent transactions so just drop it.

Fixes #3061
2025-09-12 21:19:34 +03:00
Preston Thorpe
b09dcceeef Merge 'Fixes views' from Glauber Costa
This is a collection of fixes for materialized views ahead of adding
support for JOINs.
It is mostly issues with how we assume there is a single table, with a
single delta, but we have to send more than one.
Those are things that are just objectively wrong, so I am sending it
separately to make the JOIN PR smaller.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3009
2025-09-12 07:43:32 -04:00
Pekka Enberg
06371d8894 Merge 'Add BEGIN CONCURRENT support for MVCC mode' from Pekka Enberg
Currently, when MVCC is enabled, every transaction mode supports
concurrent reads and writes, which makes it hard to adopt for existing
applications that use `BEGIN DEFERRED` or `BEGIN IMMEDIATE`.
Therefore, add support for `BEGIN CONCURRENT` transactions when MVCC is
enabled. The transaction mode allows multiple concurrent read/write
transactions that don't block each other, with conflicts resolved at
commit time. Furthermore, implement the correct semantics for `BEGIN
DEFERRED` and `BEGIN IMMEDIATE` by taking advantage of the pager level
write lock when transaction upgrades to write. This means that now
concurrent MVCC transactions are serialized against the legacy ones when
needed.
The implementation includes:
- Parser support for CONCURRENT keyword in BEGIN statements
- New Concurrent variant in TransactionMode to distinguish from regular
read/write transactions
- MVCC store tracking of exclusive transactions to support IMMEDIATE and
EXCLUSIVE modes alongside CONCURRENT
- Proper transaction state management for all transaction types in MVCC
This enables better concurrency for applications that can handle
optimistic concurrency control, while still supporting traditional
SQLite transaction semantics via IMMEDIATE and EXCLUSIVE modes.

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #3021
2025-09-12 07:38:53 +03:00
Pekka Enberg
7d8a1a0d5f Merge 'whopper: A new DST with concurrency' from Pekka Enberg
Our simulator is currently limited to concurrency of one. This
introduces a much less sophisticated DST with focus on finding
concurrency bugs.

Closes #2985
2025-09-11 18:42:45 +03:00
Levy A.
72fddccca7 fix: use sqlite compatible math functions 2025-09-11 12:36:30 -03:00
Pekka Enberg
433b60555f Add BEGIN CONCURRENT support for MVCC mode
Currently, when MVCC is enabled, every transaction mode supports
concurrent reads and writes, which makes it hard to adopt for existing
applications that use `BEGIN DEFERRED` or `BEGIN IMMEDIATE`.

Therefore, add support for `BEGIN CONCURRENT` transactions when MVCC is
enabled. The transaction mode allows multiple concurrent read/write
transactions that don't block each other, with conflicts resolved at
commit time. Furthermore, implement the correct semantics for `BEGIN
DEFERRED` and `BEGIN IMMEDIATE` by taking advantage of the pager level
write lock when transaction upgrades to write. This means that now
concurrent MVCC transactions are serialized against the legacy ones when
needed.

The implementation includes:

- Parser support for CONCURRENT keyword in BEGIN statements

- New Concurrent variant in TransactionMode to distinguish from regular
  read/write transactions

- MVCC store tracking of exclusive transactions to support IMMEDIATE and
  EXCLUSIVE modes alongside CONCURRENT

- Proper transaction state management for all transaction types in MVCC

This enables better concurrency for applications that can handle
optimistic concurrency control, while still supporting traditional
SQLite transaction semantics via IMMEDIATE and EXCLUSIVE modes.
2025-09-11 16:05:52 +03:00
Glauber Costa
874047276e views: pass a DeltaSet for merge_delta
A DeltaSet is a collection of Deltas, one per table.
We'll need that for joins. The populate step for now will still generate
a single set. That will be our next step to fix.
2025-09-11 05:30:46 -07:00
Jussi Saurio
dc9fc8c0c6 Merge 'Fix value conversion for function parameters' from Levy A.
Value conversion to float for math functions work in a more strict way
than general numeric conversion. For example, valid prefixes that can be
converted to a integer, like `"44s"` will be converted to `Value::Null`
instead of trying to recover like the math operators.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3012
2025-09-11 13:34:07 +03:00
Pekka Enberg
b572366a2b core/vbe: Demote op_transaction() logging to debug 2025-09-11 08:35:18 +03:00