Commit Graph

5423 Commits

Author SHA1 Message Date
Preston Thorpe
6fd2ad2f5e Merge 'support multiple conflict clauses in upsert' from Nikita Sivukhin
This PR implements support for `ON CONFLICT` clause chain, e.g.
```
INSERT INTO ct(id, x, y) VALUES (4, 'x', 'y1'), (5, 'a1', 'b'), (3, '_', '_')
  ON CONFLICT(x) DO UPDATE SET x = excluded.x || '-' || x, y = excluded.y || '@' || y, z = 'x' 
  ON CONFLICT(y) DO UPDATE SET x = excluded.x || '+' || x, y = excluded.y || '!' || y, z = 'y' 
  ON CONFLICT DO UPDATE SET x = excluded.x || '#' || x, y = excluded.y || '%' || y, z = 'fallback';
```

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3453
2025-09-30 19:50:59 -04:00
Pekka Enberg
229d96abf2 Merge 'core/vdbe: Don't clear parameters in Statement::reset()' from Pekka Enberg
As per SQLite API, sqlite3_reset() does *not* clear bind parameters.
Instead they're persistent across statement reset and only cleared with
sqlite3_clear_bindings().

Reviewed-by: Avinash Sajjanshetty (@avinassh)

Closes #3466
2025-09-30 21:57:59 +03:00
Nikita Sivukhin
f4263bf472 fix clippy 2025-09-30 22:43:58 +04:00
Nikita Sivukhin
9ef05adc5e fix upsert conflict handling 2025-09-30 22:39:55 +04:00
Pekka Enberg
25ffd4f01e core/vdbe: Don't clear parameters in Statement::reset()
As per SQLite API, sqlite3_reset() does *not* clear bind parameters.
Instead they're persistent across statement reset and only cleared with
sqlite3_clear_bindings().
2025-09-30 20:22:09 +03:00
Nikita Sivukhin
73f68dfcfb remove unnecessary log 2025-09-30 20:47:39 +04:00
Nikita Sivukhin
f6d829f52d simplify upsert codegen 2025-09-30 20:47:39 +04:00
Nikita Sivukhin
3590f9882d support multiple conflict clauses in upsert 2025-09-30 20:47:39 +04:00
Preston Thorpe
3456d61ac0 Merge 'Index search fixes' from Nikita Sivukhin
This PR bundles 2 fixes:
1. Index search must skip NULL values
2. UPDATE must avoid using index which column is used in the SET clause
    * This was an optimization to not do full scan in case of `UPDATE t
SET ... WHERE col = ?` but instead of doing this hacks we must properly
load updated row set to the ephemeral index and flush it after update
will be finished instead of modifying BTree inplace
    * So, for now we completely remove this optimization and quitely
wait for proper optimization to land

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3459
2025-09-30 12:34:52 -04:00
Pekka Enberg
b511b23e70 Merge 'Make encryption opt in via flag' from Avinash Sajjanshetty
We had encryption feature behind a compiler flag. However, it wasn't
enabled by default. This patch:
- enables compiler flag by default
- it also adds an opt in runtime flag `experimental-encryption`
- the runtime flag is disabled by default

Closes #3457
2025-09-30 19:31:28 +03:00
Nikita Sivukhin
c84486c411 clippy logged in as jussi - so I need to fix more stuff 2025-09-30 18:45:17 +04:00
Nikita Sivukhin
bf5567de35 fix clippy
- the proper fix is to nuke it actually :)
2025-09-30 18:06:42 +04:00
Nikita Sivukhin
4a9309fe31 fix clippy 2025-09-30 17:58:12 +04:00
Nikita Sivukhin
f1597dea90 fix all combinations of iteration direction and index order to properly handle nulls 2025-09-30 17:57:03 +04:00
Avinash Sajjanshetty
a360efa6e0 enable encryption feature flag by default 2025-09-30 19:04:25 +05:30
Nikita Sivukhin
c211fd1359 handle btree-table search properly
- btree-table doesn't have nulls in keys - so seek operation do some conversions and we shouldn't emit SeekGT { Null } in this case
2025-09-30 17:05:39 +04:00
Pekka Enberg
9788f6d005 Merge 'core/mvcc: Optimize exclusive transaction check' from Pekka Enberg
The check is in fastpath so switch to an atomic instead.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3458
2025-09-30 16:02:42 +03:00
Avinash Sajjanshetty
c8111f9555 Put encryption behind an opt in (runtime) flag 2025-09-30 18:29:18 +05:30
Jussi Saurio
81e7c26f55 Merge 'Anonymous params fix' from Nikita Sivukhin
This PR auto-assign ids for anonymous variables straight into parser.
Otherwise - it's pretty easy to mess up with traversal order in the core
code and assign ids incorrectly.
For example, before the fix, following code worked incorrectly because
parameter values were assigned first to conflict clause instead of
values:
```rs
let mut stmt = conn.prepare("INSERT INTO test VALUES (?, ?), (?, ?) ON CONFLICT DO UPDATE SET v = ?")?;
stmt.bind_at(1.try_into()?, Value::Integer(1));
stmt.bind_at(2.try_into()?, Value::Integer(20));
stmt.bind_at(3.try_into()?, Value::Integer(3));
stmt.bind_at(4.try_into()?, Value::Integer(40));
stmt.bind_at(5.try_into()?, Value::Integer(66));
```

Closes #3455
2025-09-30 15:48:35 +03:00
Nikita Sivukhin
a32ed53bd8 remove optimization
- even if index search will return only 1 row - it will call next in the loop - and we incorrecty can process same row values multiple times
- the following query failed with this optimization:

turso> CREATE TABLE t (id INTEGER PRIMARY KEY AUTOINCREMENT, k TEXT, c0 INT);
turso> CREATE UNIQUE INDEX idx_p1_0 ON t(c0);
turso> insert into t values (null, 'uu', -1);
turso> insert into t values (null, 'uu', -2);
turso> UPDATE t SET c0 = NULL WHERE c0 = -1;
turso> SELECT * FROM t
┌────┬────┬────┐
│ id │ k  │ c0 │
├────┼────┼────┤
│  1 │ uu │    │
├────┼────┼────┤
│  2 │ uu │    │
└────┴────┴────┘
2025-09-30 16:37:41 +04:00
Nikita Sivukhin
e9b8b0265d skip NULL in case of search over index 2025-09-30 16:16:04 +04:00
Pekka Enberg
3d327ba63c core/mvcc: Optimize exclusive transaction check
The check is in fastpath so switch to an atomic instead.
2025-09-30 15:00:24 +03:00
Nikita Sivukhin
e111226f3b add comment 2025-09-30 15:28:50 +04:00
Nikita Sivukhin
c955487c5f remove unnecessary enum variant 2025-09-30 14:52:59 +04:00
Nikita Sivukhin
8f005b31f4 fix bug in parameters binding 2025-09-30 14:52:30 +04:00
Nikita Sivukhin
ab92102cd8 remove parameter id assign logic from core 2025-09-30 13:58:59 +04:00
Jussi Saurio
35b584f050 Merge 'core: change root_page to i64' from Pere Diaz Bou
Closes #3454
2025-09-30 12:50:23 +03:00
Jussi Saurio
2039120106 Merge 'core/storage: Remove unused import from encryption.rs' from Pekka Enberg
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3451
2025-09-30 12:45:40 +03:00
Pere Diaz Bou
2fff6bb119 core: page id to usize 2025-09-30 11:35:06 +02:00
Pekka Enberg
1b991156f3 Merge 'core/vdbe: Fix BEGIN after BEGIN CONCURRENT check' from Pekka Enberg
We're supposed to error out only when "is_begin" is true.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3450
2025-09-30 11:34:40 +03:00
Pekka Enberg
f8a9bb1158 core/storage: Remove unused import from encryption.rs 2025-09-30 11:13:35 +03:00
Pekka Enberg
9b83fe7abf core/vdbe: Fix BEGIN after BEGIN CONCURRENT check
We're supposed to error out only when "is_begin" is true.
2025-09-30 10:55:13 +03:00
Jussi Saurio
9681377c51 Merge 'sum() can throw integer overflow' from Duy Dang
close #3311

Closes #3416
2025-09-30 10:38:37 +03:00
Jussi Saurio
594bdce999 Merge 'sum should identify if there is num in strings/prefix of strings' from Pavan Nambi
closes #3285
maybe adding seperate func for it is stupid but i kept running into
issues with not closing some random `}` and it got annoying real quick
so i just moved tht into its own func. - and as i am using same logic in
3 places i think it's ok.

Closes #3412
2025-09-30 10:37:17 +03:00
Jussi Saurio
e6a2e2a9cf Merge 'Remove double-quoted identifier assert' from Diego Reis
Closes #3301
Not every identifier should be double-quoted

Closes #3440
2025-09-30 10:34:49 +03:00
Jussi Saurio
568b0eeff3 Merge 'substr scalar should also work with non-text values' from Diego Reis
Closes #3306
Not only blobs, but numbers are also cast to text

Closes #3439
2025-09-30 10:33:40 +03:00
Pekka Enberg
2b59863447 Merge 'core/printf: Compatibility tests and fixes for printf()' from Luiz Gustavo
Addition of compatibilty tests for `printf()`.
While doing this I found some differences in the current implementation,
so this fixes those too.

Closes #3438
2025-09-30 07:44:35 +03:00
Pekka Enberg
9a08fb9e43 core/translate: Remove useless comment from logical.rs 2025-09-30 07:35:12 +03:00
Pekka Enberg
6053bb6556 Merge 'Fix materialized views with complex expressions' from Glauber Costa
SQLite supports complex expressions in group by columns - because of
course it does...
So we need to make sure that a column is created for this expression if
it doesn't exist already, and compute it, the same way we compute pre-
projections in the filter operator.
Fixes #3363
Fixes #3366
Fixes #3365

Closes #3429
2025-09-30 07:34:51 +03:00
luizgfc
3093371f76 core/printf: Fix printf compatibility with %f, %x and %o substitutions 2025-09-30 01:02:09 -03:00
Diego Reis
90f4d69774 fix(3301): Remove identifier assert assumption
Not every identifier should be double-quoted
2025-09-29 22:33:21 -03:00
Diego Reis
c9421e034d fix(3306): substr scalar should also work with non-text values 2025-09-29 21:42:21 -03:00
luizgfc
5da6ed7499 core/printf: Fix null %s formatting behavior for sqlite compatibility 2025-09-29 21:26:47 -03:00
Pekka Enberg
75f088740d Merge 'core: Disallow CREATE INDEX when MVCC is enabled' from Pekka Enberg
MVCC does currently not support indexes. Therefore,
- Fail if a database with indexes is opened with MVCC
- Disallow `CREATE INDEX` when MVCC is enabled
Fixes: #3108

Closes #3426
2025-09-29 20:53:16 +03:00
Glauber Costa
2fde976605 Fix materialized views with complex expressions
SQLite supports complex expressions in group by columns - because of
course it does...

So we need to make sure that a column is created for this expression if
it doesn't exist already, and compute it, the same way we compute
pre-projections in the filter operator.

Fixes #3363
Fixes #3366
Fixes #3365
2025-09-29 11:56:21 -05:00
Pere Diaz Bou
af98067ff1 fmt 2025-09-29 18:40:17 +02:00
Pere Diaz Bou
0f631101df core: change page idx type from usize to i64
MVCC is like the annoying younger cousin (I know because I was him) that
needs to be treated differently. MVCC requires us to use root_pages that
might not be allocated yet, and the plan is to use negative root_pages
for that case. Therefore, we need i64 in order to fit this change.
2025-09-29 18:38:43 +02:00
Pekka Enberg
05d8cca75c core: Disallow CREATE INDEX when MVCC is enabled
MVCC does currently not support indexes. Therefore,

- Fail if a database with indexes is opened with MVCC
- Disallow `CREATE INDEX` when MVCC is enabled

Fixes: #3108
2025-09-29 19:38:08 +03:00
Preston Thorpe
cdab174350 Merge 'Fix column fetch in joins' from Glauber Costa
In comparisons for joins, we were assuming that the left column belonged
to the left join (and vice-versa). That is incorrect, because you can
write the comparison condition in any order.
Fixes #3368

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3400
2025-09-29 12:34:45 -04:00
Preston Thorpe
da599a1fb8 Merge 'quoting fix' from Nikita Sivukhin
This PR moves part of string normalization to the parser layer.
Now, we dequote and unescape values in the parser, but we still need to
lowercase them for proper ignore-case comparison logic in the planner.
The reason to not lowercase in the parser is following:
1. SQLite (and tursodb) have ident->string conversion rule and by
lowercasing value early we will loose original representation
2. Some things like column names are preserve the case right now and we
better to not change this behaviour.

Closes #3344
2025-09-29 12:33:42 -04:00