Commit Graph

10542 Commits

Author SHA1 Message Date
Nikita Sivukhin
e7f6b3cd4c slightly adjust test 2025-10-27 17:00:56 +04:00
Nikita Sivukhin
05f0ee6a72 add more integration in order to properly skip backing_btree index_method 2025-10-27 17:00:26 +04:00
Nikita Sivukhin
bdbfac20fb resolve index method parameters 2025-10-27 16:39:22 +04:00
Nikita Sivukhin
a151770cea add minimal support of index_methods in the query planner in order to make integration tests work 2025-10-27 16:34:49 +04:00
Nikita Sivukhin
97dcc0869e register index_methods as db builtin extensions 2025-10-27 16:31:31 +04:00
Nikita Sivukhin
cb11417883 add index_method trait and implement simple inverted index for sparse vectors 2025-10-27 16:22:52 +04:00
Nikita Sivukhin
5d81f8db13 add simple test for index_method API 2025-10-27 16:15:50 +04:00
Nikita Sivukhin
22fe9452ac remove unnecessary parameter from integration tests 2025-10-27 15:16:12 +04:00
Nikita Sivukhin
8a80e8b743 rename custom modules to index_method like in postgresql 2025-10-27 13:18:18 +04:00
Nikita Sivukhin
408ca235d1 small refactoring 2025-10-27 12:43:38 +04:00
Nikita Sivukhin
299533b7b6 hide custom modules syntax behind --experimental-custom-modules flag 2025-10-27 12:29:05 +04:00
Nikita Sivukhin
67e62fd6ea support USING ... WITH ... syntax for index creation 2025-10-27 12:13:43 +04:00
Nikita Sivukhin
f178daa373 update comment 2025-10-27 11:47:25 +04:00
Nikita Sivukhin
906bbdd1c4 support deep nestedness 2025-10-27 11:37:42 +04:00
Pekka Enberg
1fb1fbf210 Merge 'Tighten Nyrkio p-value to 0.00001' from Henrik Ingo
This will produce even less alerts than so far, but still catches actual
changes in performance.

Closes #3840
2025-10-27 08:21:51 +02:00
Henrik Ingo
54a9821bcf Tighten Nyrkio p-value to 0.00001
This will produce even less alerts than so far, but still catches
actual changes in performance.
2025-10-27 07:09:02 +02:00
Pekka Enberg
7d035f27d8 Merge 'Strict numeric cast for op_must_be_int' from bit-aloo
closes: #3302

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3771
2025-10-26 16:42:35 +02:00
Pekka Enberg
6603f5318a Merge 'core/vdbe: Reuse cursor in op_open_write()' from Pekka Enberg
This optimization reuses an existing cursor when op_open_write() is
called on the same table/index (same root_page). This is safe because
the cursor position doesn't matter - op_rewind() is always called after
op_open_write() to position the cursor at the beginning of the
table/index before any operations are performed.
This change speeds up op_open_write() by avoiding unnecessary cursor re-
initialization.

Closes #3815
2025-10-26 12:29:20 +02:00
Pekka Enberg
ca073b5ecd Merge 'core: Switch RwLock<Arc<Pager>> to ArcSwap<Pager>' from Pekka Enberg
We don't actually need the RwLock locking capabilities, just the ability
to swap the instance.

Closes #3814
2025-10-26 12:29:11 +02:00
Pekka Enberg
6020b3d1ec Merge 'Always returns Floats for sum and avg on DBSP aggregations' from Glauber Costa
Trying to return integer sometimes to match SQLite led to more problems
that I anticipated. The reason being, we can't *really* match SQLite's
behavior unless we know the type of *every* element in the sum. This is
not impossible, but it is very hard, for very little gain.
Fixes #3831

Closes #3832
2025-10-26 12:28:18 +02:00
Pekka Enberg
77c311e21e Merge 'Update the write_varint method to use an encoded buffer of size 9 instead of 10.' from
The SQLite varint specification states that the varint is guaranteed to
be a maximum of 9 bytes, but our version of write_varint initializes a
buffer of 10 bytes. Changing the size to match the specification.
We could put an assert to check that the 10th byte is never written to
but, reducing the size  seems like the correct thing to do based on the
specification.
Varint constraints are mentioned in this specification:
https://www.sqlite.org/fileformat.html

Closes #3835
2025-10-26 12:27:09 +02:00
Sumit Patel
7f8f1bc074 Update the write_varint method to use an encoded buffer of size 9 instead of 10.
The SQLite varint specification states that the varint is guaranteed to be a maximum of 9 bytes, but our version of write_varint initializes a buffer of 10 bytes. Changing the size to match the specification.
2025-10-25 16:53:59 +05:30
Pekka Enberg
c2ac8ecc71 Merge 'sqlite3: Add multi-statement support for sqlite3_exec()' from Preston Thorpe
closes https://github.com/tursodatabase/turso/issues/3576

Closes #3809
2025-10-25 08:41:11 +03:00
Glauber Costa
1ccd61088e Always returns Floats for sum and avg on DBSP aggregations
Trying to return integer sometimes to match SQLite led to more problems
that I anticipated. The reason being, we can't *really* match SQLite's
behavior unless we know the type of *every* element in the sum. This is
not impossible, but it is very hard, for very little gain.

Fixes #3831
2025-10-24 14:13:53 -05:00
Pekka Enberg
f85ba9198f Merge 'Add DISTINCT support to aggregate operator' from Glauber Costa
Implements COUNT/SUM/AVG(DISTINCT) and SELECT DISTINCT for materialized
views. To do this we have to keep a list of the actual distinct values
(similarly to how we do for min/max). We then update the operator (and
issue deltas) only when there is a state transition (for example, if we
already count the value x = 1, and we see an insert for x = 1, we do
nothing).
SELECT DISTINCT (with no aggregator) is similar. We already have to keep
a list of the values anyway to power the aggregates. So we just issue
new deltas based on the transition, without updating the aggregator.

Closes #3808
2025-10-24 18:47:11 +03:00
PThorpe92
e9f1a451a2 Remove sqlite comment from sqlite3_exec api 2025-10-24 09:36:06 -04:00
PThorpe92
5318af16b5 Update tests in sqlite3 package to adapt to sqlite behavior 2025-10-24 09:36:06 -04:00
PThorpe92
1204494068 Fix sqlite_exec callback handling to discard rows when not provided 2025-10-24 09:36:06 -04:00
PThorpe92
ec30aad015 Replace inefficient is_query_statement fn in sqlite3 api 2025-10-24 09:36:05 -04:00
PThorpe92
fb26b72b1a Add comment from sqlite3.h describing behavior of sqlite3_exec 2025-10-24 09:36:05 -04:00
PThorpe92
921f2e72bd Add integration tests for sqlite3_exec multi-statements 2025-10-24 09:36:05 -04:00
PThorpe92
d0fd258ab5 Handle multiple statements via sqlite3_exec API 2025-10-24 09:36:05 -04:00
Jussi Saurio
8c6a6f0aa1 Merge 'Fix foreign key constraint enforcement on UNIQUE indexes' from Jussi Saurio
Our foreign key constraint checks were checking for changes in PRIMARY
KEYs, but not unique indexes - which are in practice the same thing,
apart from the `INTEGER PRIMARY KEY` special case, where the PRIMARY KEY
is an alias for the rowid of the table.
Closes #3648
Closes #3652 (reimplements)

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3825
2025-10-24 15:14:03 +03:00
Pekka Enberg
c3fb867173 core: Switch RwLock<Arc<Pager>> to ArcSwap<Pager>
We don't actually need the RwLock locking capabilities, just the ability
to swap the instance.
2025-10-24 14:10:08 +03:00
Pekka Enberg
ae60b78d82 Merge 'Switch to SQLite's Julian date logic' from Pekka Enberg
The `julian_day_converter` crate is GPL, which is problematic for apps
embedding Turso.

Closes #3822
2025-10-24 13:38:17 +03:00
bit-aloo
b2769afffd add test 2025-10-24 16:08:15 +05:30
bit-aloo
64bbca9e12 Fix op_must_be_int to use strict numeric cast 2025-10-24 16:08:15 +05:30
Pekka Enberg
e8f009af27 Merge 'docs: Add vector search section to database manual' from Pekka Enberg
Reviewed-by: Nikita Sivukhin (@sivukhin)

Closes #3824
2025-10-24 13:38:04 +03:00
Pekka Enberg
ff83b8218a docs: Add vector search section to database manual 2025-10-24 11:04:47 +03:00
Jussi Saurio
18e6a23f23 Fix foreign key constraint enforcement on UNIQUE indexes
Closes #3648

Co-authored-by: Pavan-Nambi <pavannambi999@gmail.com>
2025-10-24 11:03:55 +03:00
Pekka Enberg
5f0bbf1ce5 Merge 'bindings/javascript: Improve open error messages' from Pekka Enberg
Include database path name for debuggability.

Closes #3818
2025-10-24 09:16:48 +03:00
Pekka Enberg
827b646c24 Switch to SQLite's Julian date logic
The `julian_day_converter` crate is GPL, which is problematic for apps
embedding Turso. Switch to SQLite's Julian date logic by porting the C
code to Rust.
2025-10-24 08:31:28 +03:00
Pekka Enberg
fc0f83ea4e Merge 'perf/throughput: Use connection per transaction in rusqlite benchmark' from Pekka Enberg
Open a connection per transaction in the rusqlite benchmark so that
we're comparing the same workload with Turso.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3816
2025-10-23 20:57:04 +03:00
Pekka Enberg
4c59f29931 Merge 'core/storage: Fix WAL already enabled issue' from Pekka Enberg
If WAL is already enabled, let's just continue execution instead of
erroring out.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #3819
2025-10-23 20:56:57 +03:00
Preston Thorpe
a024265d23 Merge 'Return null terminated strings from sqlite3_column_text' from Preston Thorpe
closes #3811
adds `text_cache` which owns the null terminated bytes, which get cached
if a subsequent call to `sqlite3_column_text` is made.
#3809 depends on this fix

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3817
2025-10-23 13:21:12 -04:00
Pekka Enberg
30d183c58f bindings/javascript: Improve open error messages
Include database path name for debuggability.
2025-10-23 19:42:01 +03:00
Pekka Enberg
87069fde93 core/storage: Fix WAL already enabled issue
If WAL is already enabled, let's just continue execution instead of erroring out.
2025-10-23 19:35:46 +03:00
PThorpe92
8ed4e7cac1 Add test for null terminated string from sqlite3_column_text 2025-10-23 10:54:19 -04:00
PThorpe92
23cddbcad9 Return null terminated strings from sqlite3_column_text 2025-10-23 10:13:42 -04:00
Pekka Enberg
413c582b41 core/vdbe: Reuse cursor in op_open_write()
This optimization reuses an existing cursor when op_open_write() is
called on the same table/index (same root_page). This is safe because
the cursor position doesn't matter - op_rewind() is always called
after op_open_write() to position the cursor at the beginning of the
table/index before any operations are performed.

This change speeds up op_open_write() by avoiding unnecessary cursor
re-initialization.
2025-10-23 16:34:42 +03:00