Commit Graph

10673 Commits

Author SHA1 Message Date
Jussi Saurio
9b62687c41 Change unwrap_parens() to return Parenthesized as is, if it contains multiple values 2025-10-27 16:01:39 +02:00
Jussi Saurio
580333ddd3 Add NonFromClauseSubquery struct and add a Vec of them to SelectPlan 2025-10-27 16:01:39 +02:00
Jussi Saurio
609d9957c1 Add new QueryDestination variants for subquery types 2025-10-27 16:01:39 +02:00
Jussi Saurio
5bd6e033e6 Rename emit_subquery() to emit_from_clause_subquery() to disambiguate 2025-10-27 16:01:39 +02:00
Jussi Saurio
5eb74ce8e6 AST: Add Expr::SubqueryResult variant and enum SubqueryType 2025-10-27 16:01:39 +02:00
Nikita Sivukhin
e7f6b3cd4c slightly adjust test 2025-10-27 17:00:56 +04:00
Nikita Sivukhin
05f0ee6a72 add more integration in order to properly skip backing_btree index_method 2025-10-27 17:00:26 +04:00
Nikita Sivukhin
bdbfac20fb resolve index method parameters 2025-10-27 16:39:22 +04:00
Nikita Sivukhin
a151770cea add minimal support of index_methods in the query planner in order to make integration tests work 2025-10-27 16:34:49 +04:00
Nikita Sivukhin
97dcc0869e register index_methods as db builtin extensions 2025-10-27 16:31:31 +04:00
Nikita Sivukhin
cb11417883 add index_method trait and implement simple inverted index for sparse vectors 2025-10-27 16:22:52 +04:00
Nikita Sivukhin
5d81f8db13 add simple test for index_method API 2025-10-27 16:15:50 +04:00
Jussi Saurio
e7aa7ee2ff ProgramBuilder: add a few utility methods needed for correlated subqueries 2025-10-27 14:03:41 +02:00
Jussi Saurio
5c05383cc1 Implement union for ColumnUsedMask 2025-10-27 13:57:56 +02:00
Jussi Saurio
3a1d6d8879 Improve error messages in translate_expr()
The current error messages are misleading, as the user may encounter
these errors in expressions outside the WHERE clause, too.
2025-10-27 13:51:59 +02:00
Jussi Saurio
de81af29e5 find_table_by_internal_id() returns whether table is an outer query reference
Unfortunately, our current translation machinery is unable to know for sure
whether a subquery reference to an outer table 't1' has opened a table cursor,
an index cursor, or both.

For this reason, return a flag from `TableReferences::find_table_by_internal_id()`
that tells the caller whether the table is an outer query reference, and further
commits will have some additional logic to decide which cursor a subquery will
read from when referencing a table from the outer query.
2025-10-27 13:47:49 +02:00
Jussi Saurio
c0c425b5d6 EXPLAIN: indent BeginSubrtn...Return blocks properly
WHERE clause subqueries use the BeginSubrtn instruction.

The corresponding closing instruction for BeginSubrtn is Return,
but Return is also used for other purposes, so we need to track pairs of
BeginSubrtn and Return that share the same 1st parameter (the subroutine register),
so that the EXPLAIN output for those subroutine contents is indented properly.
2025-10-27 13:42:00 +02:00
Nikita Sivukhin
22fe9452ac remove unnecessary parameter from integration tests 2025-10-27 15:16:12 +04:00
Nikita Sivukhin
8a80e8b743 rename custom modules to index_method like in postgresql 2025-10-27 13:18:18 +04:00
Nikita Sivukhin
408ca235d1 small refactoring 2025-10-27 12:43:38 +04:00
Nikita Sivukhin
299533b7b6 hide custom modules syntax behind --experimental-custom-modules flag 2025-10-27 12:29:05 +04:00
Nikita Sivukhin
67e62fd6ea support USING ... WITH ... syntax for index creation 2025-10-27 12:13:43 +04:00
Nikita Sivukhin
f178daa373 update comment 2025-10-27 11:47:25 +04:00
Nikita Sivukhin
906bbdd1c4 support deep nestedness 2025-10-27 11:37:42 +04:00
Pekka Enberg
1fb1fbf210 Merge 'Tighten Nyrkio p-value to 0.00001' from Henrik Ingo
This will produce even less alerts than so far, but still catches actual
changes in performance.

Closes #3840
2025-10-27 08:21:51 +02:00
Henrik Ingo
54a9821bcf Tighten Nyrkio p-value to 0.00001
This will produce even less alerts than so far, but still catches
actual changes in performance.
2025-10-27 07:09:02 +02:00
kimminseok
2b456ec7e4 chore: apply spotless formatting 2025-10-27 00:31:49 +09:00
Pekka Enberg
2429159e5d bindings/rust: Enable mimalloc as global allocator
This improves performance by using mimalloc for memory allocation in the
Rust bindings.
2025-10-26 16:55:33 +02:00
Pekka Enberg
dfab8c44bc core: Switch to FxHash to improve performance
The default Rust hash map is slow for integer keys. Switch to FxHash
instead to reduce executed instructions for, for example, throughput
benchmark.

Note that dirty page tracking is changed to BTreeMap to ensure that the
hash function changes don't impact the WAL frame order, which SQLite
guarantees to be page number ordered.

Before:

```
penberg@turing:~/src/tursodatabase/turso/perf/throughput/turso$ perf stat ../../../target/release/write-throughput --threads 1 --batch-size 100 --compute 0 -i 10000
Turso,1,100,0,106875.21

 Performance counter stats for '../../../target/release/write-throughput --threads 1 --batch-size 100 --compute 0 -i 10000':

          2,908.02 msec task-clock                       #    0.310 CPUs utilized
            30,508      context-switches                 #   10.491 K/sec
               261      cpu-migrations                   #   89.752 /sec
               813      page-faults                      #  279.572 /sec
    20,655,313,128      instructions                     #    1.73  insn per cycle
                                                  #    0.14  stalled cycles per insn
    11,930,088,949      cycles                           #    4.102 GHz
     2,845,040,381      stalled-cycles-frontend          #   23.85% frontend cycles idle
     3,814,652,892      branches                         #    1.312 G/sec
        54,760,600      branch-misses                    #    1.44% of all branches

       9.372979876 seconds time elapsed

       2.276835000 seconds user
       0.530135000 seconds sys
```

After:

```
penberg@turing:~/src/tursodatabase/turso/perf/throughput/turso$ perf stat ../../../target/release/write-throughput --threads 1 --batch-size 100 --compute 0 -i 10000
Turso,1,100,0,108663.84

 Performance counter stats for '../../../target/release/write-throughput --threads 1 --batch-size 100 --compute 0 -i 10000':

          2,838.65 msec task-clock                       #    0.308 CPUs utilized
            30,629      context-switches                 #   10.790 K/sec
               351      cpu-migrations                   #  123.650 /sec
               818      page-faults                      #  288.165 /sec
    19,887,102,451      instructions                     #    1.72  insn per cycle
                                                  #    0.14  stalled cycles per insn
    11,593,166,024      cycles                           #    4.084 GHz
     2,830,298,617      stalled-cycles-frontend          #   24.41% frontend cycles idle
     3,764,334,333      branches                         #    1.326 G/sec
        53,157,766      branch-misses                    #    1.41% of all branches

       9.218225731 seconds time elapsed

       2.231889000 seconds user
       0.508785000 seconds sys

```
2025-10-26 16:48:59 +02:00
Pekka Enberg
7d035f27d8 Merge 'Strict numeric cast for op_must_be_int' from bit-aloo
closes: #3302

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3771
2025-10-26 16:42:35 +02:00
김민석
24181ad307 Merge branch 'main' into feature/result-set-was-null 2025-10-26 23:02:36 +09:00
kimminseok
17f1a070ed Add comprehensive tests for ResultSet getter methods 2025-10-26 22:37:47 +09:00
kimminseok
71f53b5850 Add wasNull tracking to getObject methods 2025-10-26 22:36:38 +09:00
kimminseok
27233a947f Add consistent wasNull handling for stream getter methods 2025-10-26 22:34:43 +09:00
kimminseok
7ee339a754 Extract timezone offset calculation to helper method 2025-10-26 22:09:47 +09:00
kimminseok
791e19892d Refactor columnLabel getters to use delegation pattern 2025-10-26 21:45:54 +09:00
kimminseok
5f10e647e3 Fix wasNull not being set in getter methods 2025-10-26 21:31:11 +09:00
Pekka Enberg
6603f5318a Merge 'core/vdbe: Reuse cursor in op_open_write()' from Pekka Enberg
This optimization reuses an existing cursor when op_open_write() is
called on the same table/index (same root_page). This is safe because
the cursor position doesn't matter - op_rewind() is always called after
op_open_write() to position the cursor at the beginning of the
table/index before any operations are performed.
This change speeds up op_open_write() by avoiding unnecessary cursor re-
initialization.

Closes #3815
2025-10-26 12:29:20 +02:00
Pekka Enberg
ca073b5ecd Merge 'core: Switch RwLock<Arc<Pager>> to ArcSwap<Pager>' from Pekka Enberg
We don't actually need the RwLock locking capabilities, just the ability
to swap the instance.

Closes #3814
2025-10-26 12:29:11 +02:00
Pekka Enberg
6020b3d1ec Merge 'Always returns Floats for sum and avg on DBSP aggregations' from Glauber Costa
Trying to return integer sometimes to match SQLite led to more problems
that I anticipated. The reason being, we can't *really* match SQLite's
behavior unless we know the type of *every* element in the sum. This is
not impossible, but it is very hard, for very little gain.
Fixes #3831

Closes #3832
2025-10-26 12:28:18 +02:00
Pekka Enberg
77c311e21e Merge 'Update the write_varint method to use an encoded buffer of size 9 instead of 10.' from
The SQLite varint specification states that the varint is guaranteed to
be a maximum of 9 bytes, but our version of write_varint initializes a
buffer of 10 bytes. Changing the size to match the specification.
We could put an assert to check that the 10th byte is never written to
but, reducing the size  seems like the correct thing to do based on the
specification.
Varint constraints are mentioned in this specification:
https://www.sqlite.org/fileformat.html

Closes #3835
2025-10-26 12:27:09 +02:00
Sumit Patel
7f8f1bc074 Update the write_varint method to use an encoded buffer of size 9 instead of 10.
The SQLite varint specification states that the varint is guaranteed to be a maximum of 9 bytes, but our version of write_varint initializes a buffer of 10 bytes. Changing the size to match the specification.
2025-10-25 16:53:59 +05:30
Pekka Enberg
c2ac8ecc71 Merge 'sqlite3: Add multi-statement support for sqlite3_exec()' from Preston Thorpe
closes https://github.com/tursodatabase/turso/issues/3576

Closes #3809
2025-10-25 08:41:11 +03:00
Glauber Costa
1ccd61088e Always returns Floats for sum and avg on DBSP aggregations
Trying to return integer sometimes to match SQLite led to more problems
that I anticipated. The reason being, we can't *really* match SQLite's
behavior unless we know the type of *every* element in the sum. This is
not impossible, but it is very hard, for very little gain.

Fixes #3831
2025-10-24 14:13:53 -05:00
Pavan-Nambi
277a989a71 fmt 2025-10-24 21:34:17 +05:30
Pavan-Nambi
7dda783006 clippy - gotta feature autovaccuum n ptrmaps 2025-10-24 21:30:34 +05:30
Pekka Enberg
f85ba9198f Merge 'Add DISTINCT support to aggregate operator' from Glauber Costa
Implements COUNT/SUM/AVG(DISTINCT) and SELECT DISTINCT for materialized
views. To do this we have to keep a list of the actual distinct values
(similarly to how we do for min/max). We then update the operator (and
issue deltas) only when there is a state transition (for example, if we
already count the value x = 1, and we see an insert for x = 1, we do
nothing).
SELECT DISTINCT (with no aggregator) is similar. We already have to keep
a list of the values anyway to power the aggregates. So we just issue
new deltas based on the transition, without updating the aggregator.

Closes #3808
2025-10-24 18:47:11 +03:00
Pavan-Nambi
cf93360b9c cleanup
Co-authored-by: Mikaël Francoeur <mikael.francoeur@hotmail.com>

cleanup

Co-authored-by: Mikaël Francoeur <mikael.francoeur@hotmail.com>
2025-10-24 19:35:13 +05:30
PThorpe92
e9f1a451a2 Remove sqlite comment from sqlite3_exec api 2025-10-24 09:36:06 -04:00
PThorpe92
5318af16b5 Update tests in sqlite3 package to adapt to sqlite behavior 2025-10-24 09:36:06 -04:00