Commit Graph

10545 Commits

Author SHA1 Message Date
Jussi Saurio
59363a1be3 Translate Expr::SubqueryResult into bytecode 2025-10-27 16:01:39 +02:00
Jussi Saurio
bc2a7c79f9 Add TODO comment about subquery positions we don't support yet 2025-10-27 16:01:39 +02:00
Jussi Saurio
8fecd82311 Emit non from clause subqueries in translation 2025-10-27 16:01:39 +02:00
Jussi Saurio
bf66999f64 Add emit_non_from_clause_subquery() method 2025-10-27 16:01:39 +02:00
Jussi Saurio
8e1987bd5d Rename emit_subqueries() to emit_from_clause_subqueries() to disambiguate 2025-10-27 16:01:39 +02:00
Jussi Saurio
58caf32fe2 Add plan_subqueries_from_where_clause() method and use it in Select planning 2025-10-27 16:01:39 +02:00
Jussi Saurio
c54988192e Add SelectPlan::is_correlated() method 2025-10-27 16:01:39 +02:00
Jussi Saurio
9b62687c41 Change unwrap_parens() to return Parenthesized as is, if it contains multiple values 2025-10-27 16:01:39 +02:00
Jussi Saurio
580333ddd3 Add NonFromClauseSubquery struct and add a Vec of them to SelectPlan 2025-10-27 16:01:39 +02:00
Jussi Saurio
609d9957c1 Add new QueryDestination variants for subquery types 2025-10-27 16:01:39 +02:00
Jussi Saurio
5bd6e033e6 Rename emit_subquery() to emit_from_clause_subquery() to disambiguate 2025-10-27 16:01:39 +02:00
Jussi Saurio
5eb74ce8e6 AST: Add Expr::SubqueryResult variant and enum SubqueryType 2025-10-27 16:01:39 +02:00
Jussi Saurio
e7aa7ee2ff ProgramBuilder: add a few utility methods needed for correlated subqueries 2025-10-27 14:03:41 +02:00
Jussi Saurio
5c05383cc1 Implement union for ColumnUsedMask 2025-10-27 13:57:56 +02:00
Jussi Saurio
3a1d6d8879 Improve error messages in translate_expr()
The current error messages are misleading, as the user may encounter
these errors in expressions outside the WHERE clause, too.
2025-10-27 13:51:59 +02:00
Jussi Saurio
de81af29e5 find_table_by_internal_id() returns whether table is an outer query reference
Unfortunately, our current translation machinery is unable to know for sure
whether a subquery reference to an outer table 't1' has opened a table cursor,
an index cursor, or both.

For this reason, return a flag from `TableReferences::find_table_by_internal_id()`
that tells the caller whether the table is an outer query reference, and further
commits will have some additional logic to decide which cursor a subquery will
read from when referencing a table from the outer query.
2025-10-27 13:47:49 +02:00
Jussi Saurio
c0c425b5d6 EXPLAIN: indent BeginSubrtn...Return blocks properly
WHERE clause subqueries use the BeginSubrtn instruction.

The corresponding closing instruction for BeginSubrtn is Return,
but Return is also used for other purposes, so we need to track pairs of
BeginSubrtn and Return that share the same 1st parameter (the subroutine register),
so that the EXPLAIN output for those subroutine contents is indented properly.
2025-10-27 13:42:00 +02:00
Pekka Enberg
1fb1fbf210 Merge 'Tighten Nyrkio p-value to 0.00001' from Henrik Ingo
This will produce even less alerts than so far, but still catches actual
changes in performance.

Closes #3840
2025-10-27 08:21:51 +02:00
Henrik Ingo
54a9821bcf Tighten Nyrkio p-value to 0.00001
This will produce even less alerts than so far, but still catches
actual changes in performance.
2025-10-27 07:09:02 +02:00
Pekka Enberg
7d035f27d8 Merge 'Strict numeric cast for op_must_be_int' from bit-aloo
closes: #3302

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3771
2025-10-26 16:42:35 +02:00
Pekka Enberg
6603f5318a Merge 'core/vdbe: Reuse cursor in op_open_write()' from Pekka Enberg
This optimization reuses an existing cursor when op_open_write() is
called on the same table/index (same root_page). This is safe because
the cursor position doesn't matter - op_rewind() is always called after
op_open_write() to position the cursor at the beginning of the
table/index before any operations are performed.
This change speeds up op_open_write() by avoiding unnecessary cursor re-
initialization.

Closes #3815
2025-10-26 12:29:20 +02:00
Pekka Enberg
ca073b5ecd Merge 'core: Switch RwLock<Arc<Pager>> to ArcSwap<Pager>' from Pekka Enberg
We don't actually need the RwLock locking capabilities, just the ability
to swap the instance.

Closes #3814
2025-10-26 12:29:11 +02:00
Pekka Enberg
6020b3d1ec Merge 'Always returns Floats for sum and avg on DBSP aggregations' from Glauber Costa
Trying to return integer sometimes to match SQLite led to more problems
that I anticipated. The reason being, we can't *really* match SQLite's
behavior unless we know the type of *every* element in the sum. This is
not impossible, but it is very hard, for very little gain.
Fixes #3831

Closes #3832
2025-10-26 12:28:18 +02:00
Pekka Enberg
77c311e21e Merge 'Update the write_varint method to use an encoded buffer of size 9 instead of 10.' from
The SQLite varint specification states that the varint is guaranteed to
be a maximum of 9 bytes, but our version of write_varint initializes a
buffer of 10 bytes. Changing the size to match the specification.
We could put an assert to check that the 10th byte is never written to
but, reducing the size  seems like the correct thing to do based on the
specification.
Varint constraints are mentioned in this specification:
https://www.sqlite.org/fileformat.html

Closes #3835
2025-10-26 12:27:09 +02:00
Sumit Patel
7f8f1bc074 Update the write_varint method to use an encoded buffer of size 9 instead of 10.
The SQLite varint specification states that the varint is guaranteed to be a maximum of 9 bytes, but our version of write_varint initializes a buffer of 10 bytes. Changing the size to match the specification.
2025-10-25 16:53:59 +05:30
Pekka Enberg
c2ac8ecc71 Merge 'sqlite3: Add multi-statement support for sqlite3_exec()' from Preston Thorpe
closes https://github.com/tursodatabase/turso/issues/3576

Closes #3809
2025-10-25 08:41:11 +03:00
Glauber Costa
1ccd61088e Always returns Floats for sum and avg on DBSP aggregations
Trying to return integer sometimes to match SQLite led to more problems
that I anticipated. The reason being, we can't *really* match SQLite's
behavior unless we know the type of *every* element in the sum. This is
not impossible, but it is very hard, for very little gain.

Fixes #3831
2025-10-24 14:13:53 -05:00
Pekka Enberg
f85ba9198f Merge 'Add DISTINCT support to aggregate operator' from Glauber Costa
Implements COUNT/SUM/AVG(DISTINCT) and SELECT DISTINCT for materialized
views. To do this we have to keep a list of the actual distinct values
(similarly to how we do for min/max). We then update the operator (and
issue deltas) only when there is a state transition (for example, if we
already count the value x = 1, and we see an insert for x = 1, we do
nothing).
SELECT DISTINCT (with no aggregator) is similar. We already have to keep
a list of the values anyway to power the aggregates. So we just issue
new deltas based on the transition, without updating the aggregator.

Closes #3808
2025-10-24 18:47:11 +03:00
PThorpe92
e9f1a451a2 Remove sqlite comment from sqlite3_exec api 2025-10-24 09:36:06 -04:00
PThorpe92
5318af16b5 Update tests in sqlite3 package to adapt to sqlite behavior 2025-10-24 09:36:06 -04:00
PThorpe92
1204494068 Fix sqlite_exec callback handling to discard rows when not provided 2025-10-24 09:36:06 -04:00
PThorpe92
ec30aad015 Replace inefficient is_query_statement fn in sqlite3 api 2025-10-24 09:36:05 -04:00
PThorpe92
fb26b72b1a Add comment from sqlite3.h describing behavior of sqlite3_exec 2025-10-24 09:36:05 -04:00
PThorpe92
921f2e72bd Add integration tests for sqlite3_exec multi-statements 2025-10-24 09:36:05 -04:00
PThorpe92
d0fd258ab5 Handle multiple statements via sqlite3_exec API 2025-10-24 09:36:05 -04:00
Jussi Saurio
8c6a6f0aa1 Merge 'Fix foreign key constraint enforcement on UNIQUE indexes' from Jussi Saurio
Our foreign key constraint checks were checking for changes in PRIMARY
KEYs, but not unique indexes - which are in practice the same thing,
apart from the `INTEGER PRIMARY KEY` special case, where the PRIMARY KEY
is an alias for the rowid of the table.
Closes #3648
Closes #3652 (reimplements)

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3825
2025-10-24 15:14:03 +03:00
Pekka Enberg
c3fb867173 core: Switch RwLock<Arc<Pager>> to ArcSwap<Pager>
We don't actually need the RwLock locking capabilities, just the ability
to swap the instance.
2025-10-24 14:10:08 +03:00
Pekka Enberg
ae60b78d82 Merge 'Switch to SQLite's Julian date logic' from Pekka Enberg
The `julian_day_converter` crate is GPL, which is problematic for apps
embedding Turso.

Closes #3822
2025-10-24 13:38:17 +03:00
bit-aloo
b2769afffd add test 2025-10-24 16:08:15 +05:30
bit-aloo
64bbca9e12 Fix op_must_be_int to use strict numeric cast 2025-10-24 16:08:15 +05:30
Pekka Enberg
e8f009af27 Merge 'docs: Add vector search section to database manual' from Pekka Enberg
Reviewed-by: Nikita Sivukhin (@sivukhin)

Closes #3824
2025-10-24 13:38:04 +03:00
Pekka Enberg
ff83b8218a docs: Add vector search section to database manual 2025-10-24 11:04:47 +03:00
Jussi Saurio
18e6a23f23 Fix foreign key constraint enforcement on UNIQUE indexes
Closes #3648

Co-authored-by: Pavan-Nambi <pavannambi999@gmail.com>
2025-10-24 11:03:55 +03:00
Pekka Enberg
5f0bbf1ce5 Merge 'bindings/javascript: Improve open error messages' from Pekka Enberg
Include database path name for debuggability.

Closes #3818
2025-10-24 09:16:48 +03:00
Pekka Enberg
827b646c24 Switch to SQLite's Julian date logic
The `julian_day_converter` crate is GPL, which is problematic for apps
embedding Turso. Switch to SQLite's Julian date logic by porting the C
code to Rust.
2025-10-24 08:31:28 +03:00
Pekka Enberg
fc0f83ea4e Merge 'perf/throughput: Use connection per transaction in rusqlite benchmark' from Pekka Enberg
Open a connection per transaction in the rusqlite benchmark so that
we're comparing the same workload with Turso.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3816
2025-10-23 20:57:04 +03:00
Pekka Enberg
4c59f29931 Merge 'core/storage: Fix WAL already enabled issue' from Pekka Enberg
If WAL is already enabled, let's just continue execution instead of
erroring out.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #3819
2025-10-23 20:56:57 +03:00
Preston Thorpe
a024265d23 Merge 'Return null terminated strings from sqlite3_column_text' from Preston Thorpe
closes #3811
adds `text_cache` which owns the null terminated bytes, which get cached
if a subsequent call to `sqlite3_column_text` is made.
#3809 depends on this fix

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3817
2025-10-23 13:21:12 -04:00
Pekka Enberg
30d183c58f bindings/javascript: Improve open error messages
Include database path name for debuggability.
2025-10-23 19:42:01 +03:00
Pekka Enberg
87069fde93 core/storage: Fix WAL already enabled issue
If WAL is already enabled, let's just continue execution instead of erroring out.
2025-10-23 19:35:46 +03:00