Commit Graph

394 Commits

Author SHA1 Message Date
Pekka Enberg
3aa885690c core/vdbe: Don't call rowid() unless view processing is enabled
Currently, the simulator complains of the following error:

```
Error: failed with error: 'attempt to multiply with overflow'
```

However, we don't enable views in the simulator so -- despite being an
issue -- we should never see this. Let's fix `op_delete()` some more not
to not even call rowid() unless view processing is enabled.
2025-08-11 10:56:58 +03:00
Pekka Enberg
6f3278f6ed core/vdbe: Perform view processing in op_delete only if views are enabled
Like in op_insert(), let's perform view processing only if views are
actually enabled.
2025-08-11 10:26:16 +03:00
Pekka Enberg
6e8d1a5c7d Merge 'Initial pass on incremental view maintenance with DBSP' from Glauber Costa
Implement very basic views using DBSP
This is just the bare minimum that I needed to convince myself that this
 approach will work. The only views that we support are slices of the
 main table: no aggregations, no joins, no projections.
 * drop view is implemented.
 * view population is implemented.
 * deletes, inserts and updates are implemented.
 much like indexes before, a flag must be passed to enable views.

Closes #2530
2025-08-11 07:45:43 +03:00
Glauber Costa
145d6eede7 Implement very basic views using DBSP
This is just the bare minimum that I needed to convince myself that this
approach will work. The only views that we support are slices of the
main table: no aggregations, no joins, no projections.

drop view is implemented.
view population is implemented.
deletes, inserts and updates are implemented.

much like indexes before, a flag must be passed to enable views.
2025-08-10 23:34:04 -05:00
Glauber Costa
e255fc9a81 Add table name to the delete bytecode
When building views (soon), it will be important to know which table
is being deleted. Getting from the cursor id is very cumbersome.

What we are doing here is symmetrical to op_insert, and sqlite also
passes table information in one of the registers (p4)
2025-08-10 22:50:25 -05:00
PThorpe92
0884fec799 Use parent buffer pool for ephemeral pager and wal 2025-08-08 10:55:26 -04:00
Levy A.
658405d6b3 feat: add AddColumn instruction 2025-08-07 11:43:16 -03:00
PThorpe92
df2c39b98e Use load_insn macro for op_journal_mode 2025-08-06 23:42:47 -04:00
Glauber Costa
071330a739 implement the JournalMode vdbe instruction
We do this already, but not through any opcode.
Move it to an opcode for compatibility reasons.
2025-08-06 19:30:19 -05:00
Glauber Costa
f36974f086 implement the MaxPgCount opcode
It is used by the pragma max_page_count, which is also implemented.
2025-08-06 13:20:15 -05:00
Levy A.
c9e1eca8dc feat: add DropColumn instruction 2025-08-06 13:39:30 -03:00
Jussi Saurio
8e4597d11b Merge 'Add load_insn macro for compiler hint in vdbe::execute hot path' from Preston Thorpe
The built-in `unreachable!` macro, believe it or not is just an alias
for `panic!` and does not actually provide the compiler with a hint that
the path is not reachable.
This provides a wrapper around the actual
`std::hint::unreachable_unchecked()`, to be used only in the very hot
path of `execute` where it is not possible to be the incorrect variant.

Closes #2459
2025-08-06 12:05:33 +03:00
PThorpe92
00a3c7eb52 Apply PR comments, fix syntax 2025-08-05 21:17:23 -04:00
PThorpe92
d5f9e60dfc Add assert_insn for compiler hint in execute hot path 2025-08-05 18:32:27 -04:00
Glauber Costa
d1be7ad0bb implement the collseq bytecode instruction
SQLite generates those in aggregations like min / max with collation
information either in the table definition or in the column expression.

We currently generate the wrong result here, and properly generating the
bytecode instruction fixes it.
2025-08-05 13:49:04 -05:00
Glauber Costa
6a66053ca8 make sure value comparisons for min and max are collation aware
They currently aren't, which isn't right.
2025-08-05 13:39:38 -05:00
Pekka Enberg
9492a29d47 Merge 'Fix performance regression' from Jussi Saurio
Closes #2440
## Fix 1
Do not start a read transaction when a SELECT is not going to access the
database, which means we can avoid checking whether the schema has
changed.
## Fix 2
Add a field `accesses_db` to `Program` and `Statement` so we can avoid
even checking for `SchemaUpdated` errors when it's not possible to get
one.
## Fix 3
Avoid doing any work in `commit_txn` when not in a transaction. This
optimization is only enabled when `mv_store.is_none()`, because MVCC has
its own logic and this doesn't work with MVCC enabled, and honestly I'm
too tired to find out why. Left an inline comment about it, though.
```sql
Execute `SELECT 1`/limbo_execute_select_1
                        time:   [21.440 ns 21.513 ns 21.586 ns]
                        change: [-60.766% -60.616% -60.453%] (p = 0.00 < 0.05)
                        Performance has improved.
```
Effect is even more dramatic in CI where the latency is down over 80%

Closes #2441
2025-08-05 16:30:18 +03:00
Jussi Saurio
c498196c7b fix/perf: fix regression in SELECT 1 benchmark
Do not start a read transaction when a SELECT is not going to access
the database, which means we can avoid checking whether the schema has
changed.
2025-08-05 15:10:55 +03:00
Pere Diaz Bou
474f0d8bbc core/mvcc: implement exists 2025-08-05 13:34:51 +02:00
Jussi Saurio
a66b56678d Merge 'Reprepare Statements when Schema changes' from Pedro Muniz
Closes #1967
To support this I had to change how we did `epilogue` similarly to how
SQLite does it. SQLIte first declares a `beginWriteOperation` when some
statement is going to necessitate a Write Transaction. And as we now
need to pass the current schema cookie to `epilogue` it was easier to
call epilogue only in one location (like we do with prologue), and just
have each statement declare their intentions separately. This allows us
to not have to pass the Schema around just to do the epilogue. I believe
this is something that @jussisaurio would be interested in.
~Also had to disable the MVCC test, as it was extremely buggy for me.~
Just disabled reprepare statements for MVCC

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2214
2025-08-05 00:01:14 +03:00
pedrocarlo
f1df9a909e state machine for 'rewind' 2025-08-04 12:59:52 -03:00
pedrocarlo
e9c3f0d55b disable schema reprepare for MVCC 2025-08-04 12:32:34 -03:00
pedrocarlo
0e3e64878c workaround the fact that to reparse schema we have to avoid falling into a reprepared statement loop 2025-08-04 12:32:34 -03:00
pedrocarlo
266a7e1c66 do not error in op_transaction if page 1 was not allocated 2025-08-04 12:32:34 -03:00
pedrocarlo
0779c23bbf fix merge conflicts 2025-08-04 12:32:34 -03:00
pedrocarlo
d2019e95f3 pass schema to epilogue for schema_version checking + do not Pragma Schema Version in open_with_flags to avoid infinite loop in reprepare. Just access the database header directly 2025-08-04 12:32:34 -03:00
pedrocarlo
c567636deb Adjust Transaction OpCode to accept schema cookie + check if cookie changed 2025-08-04 12:32:34 -03:00
Jussi Saurio
506bb5f67f Merge 'Direct schema mutation – add instruction' from Levy A.
Resolves #2378.
```
`ALTER TABLE _ RENAME TO _`/limbo_rename_table/
                        time:   [15.645 ms 15.741 ms 15.850 ms]
Found 12 outliers among 100 measurements (12.00%)
  8 (8.00%) high mild
  4 (4.00%) high severe
`ALTER TABLE _ RENAME TO _`/sqlite_rename_table/
                        time:   [34.728 ms 35.260 ms 35.955 ms]
Found 15 outliers among 100 measurements (15.00%)
  8 (8.00%) high mild
  7 (7.00%) high severe
  ```
<img width="1000" height="199" alt="image" src="https://github.com/user-
attachments/assets/ad943355-b57d-43d9-8a84-850461b8af41" />

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2399
2025-08-04 16:55:38 +03:00
Pere Diaz Bou
f26e442597 core/mvcc: fix new rowid
next rowid was being tracked globally for all tables and restarted to 0
every time database was opened
2025-08-04 12:31:17 +02:00
Levy A.
b9a3a93ef0 fix: clippy 2025-08-02 20:06:05 -03:00
Levy A.
b14a11a2fd fix: change name for schema btree + fix benchmark 2025-08-02 17:17:36 -03:00
Jussi Saurio
130e1f80ea fix/vdbe: call seek_to_last() only once in op_new_rowid 2025-08-02 14:18:58 +03:00
Pekka Enberg
2c05a3e787 Merge 'perf/vdbe: remove eager cloning in op_comparison' from Jussi Saurio
Shaves off about 100-200ms of runtime from TPC-H `19.sql`

Closes #2385
2025-08-02 10:01:47 +03:00
Jussi Saurio
be1456f7cb Merge 'use state machine for NoConflict opcode' from Mikaël Francoeur
This will save some work when yielding to IO. Previously, on every
invocation, if the record was a packed record, we parsed it and iterated
through the values to check for nulls. Now, the pre-seeking work is done
only once.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2394
2025-08-02 09:37:00 +03:00
Levy A.
1e177053cb feat: add RenameTable instruction
direct schema mutation, no reparsing
2025-08-01 21:11:25 -03:00
Mikaël Francoeur
81412b4a17 use state machine for NoConflict opcode 2025-08-01 17:29:57 -04:00
rajajisai
f6d43df46f Merge branch 'tursodatabase:main' into issue/2077 2025-08-01 15:20:36 -04:00
rajajisai
d09dd4170b Format code 2025-08-01 11:59:57 -07:00
rajajisai
30c059483e Parse value as float if it cannot be parsed as integer(when the value cannot fit in i64) 2025-08-01 10:49:40 -07:00
rajajisai
7e84148883 Fix integer overflow check in number parser 2025-08-01 10:10:02 -07:00
Jussi Saurio
d58d71ad1b perf/vdbe: remove eager cloning in op_comparison 2025-08-01 14:04:56 +03:00
Pere Diaz Bou
5ad7d10790 core/mvcc: fix use of rwlock 2025-08-01 10:38:41 +02:00
Pere Diaz Bou
c4318cac36 core/mvcc: fix tests 2025-08-01 10:38:41 +02:00
Pere Diaz Bou
49a00ff338 core/mvcc: load table's rowid on initialization
We need to load rowids into mvcc's store in order before doing any read
in case there are rows.

This has a performance penalty for now as expected because we should,
ideally, scan for row ids lazily instead.
2025-08-01 10:38:41 +02:00
Pere Diaz Bou
b399ddea1b core/mvcc: begin pager read txn on mvcc begin_txn 2025-08-01 10:38:41 +02:00
Jussi Saurio
e147494642 pager: make WAL optional again and remove DummyWAL 2025-08-01 10:14:35 +03:00
Jussi Saurio
8c6293ebb7 VDBE: use temporary on-disk file for OpenEphemeral 2025-08-01 10:14:01 +03:00
Preston Thorpe
bd9df6262f Merge 'IN queries' from Glauber Costa
Merge 'IN queries' from Glauber Costa

Implement IN queries.
It is currently as todo!(), but my main motivation is that scavenging
for EXPLAINs, that pattern, at least in simple queries like SELECT ...
IN (1,2,3) uses the AddImm instruction we just added.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2342
2025-07-31 10:00:18 -04:00
Glauber Costa
9e8ba5263b Implement the AddImm opcode
It is a simple opcode. The hard part was finding a sqlite statement
that uses it =)
2025-07-31 08:08:07 -05:00
Jussi Saurio
9e1fca2eba vdbe: disallow checkpointing in interactive tx 2025-07-31 13:16:33 +03:00