Commit Graph

10754 Commits

Author SHA1 Message Date
Pekka Enberg
7891be96fd Merge 'Refactor affinity conversions for reusability' from Pedro Muniz
Depends on #3920
Moves some code around so it is easier to reuse and less cluttered in
`execute.rs`, and changes how `compare` works. Instead of mutating some
register, we now just return the possible `ValueRef` representation of
that affinity. This allows other parts of the codebase to reuse this
logic without needing to have an owned `Value` or a `&mut Register`

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3923
2025-11-10 09:02:22 +02:00
Pekka Enberg
2be515247f Merge 'Create AsValueRef trait to allow us to be agnostic over ownership of Value or ValueRef' from Pedro Muniz
Depends on #3919
Also change `op_compare` to reuse the same compare_immutable logic
First step to finish #2304

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3920
2025-11-10 09:01:59 +02:00
Pekka Enberg
4bb0edac5e Merge 'Move value functions to separate file' from Pedro Muniz
Makes it easier to visualize what is related to Value and what is
related to opcodes. This will also facilitate in my next PR to
generalize certain function over `Value` and `ValueRef` as listed in
#2304

Closes #3919
2025-11-10 09:01:29 +02:00
Preston Thorpe
49f9b74c56 Merge 'Avoid heavy macro' from Nikita Sivukhin
Rewrite `parse_modifier` function because its current version lead to
enormous amount of generated LLVM code which significantly increase
compilation time
```sh
$> cargo llvm-lines
  Lines                  Copies               Function name
  -----                  ------               -------------
  1322611                29544                (TOTAL)
   278720 (21.1%, 21.1%)     1 (0.0%,  0.0%)  turso_core::functions::datetime::parse_modifier
```
Before:
```sh
$> cargo check
warning: `turso_core` (lib) generated 2 warnings
    Finished `dev` profile [unoptimized] target(s) in 5.61s
```
After:
```sh
$> cargo check
warning: `turso_core` (lib) generated 2 warnings
    Finished `dev` profile [unoptimized] target(s) in 2.24s
```

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3929
2025-11-09 11:37:06 -05:00
Nikita Sivukhin
6e0e4e20ab add comment 2025-11-09 16:04:16 +04:00
Nikita Sivukhin
ec91829835 rewrite parse_modifier function because its current version lead to enormous amount of generated LLVM code which significantly increase compilation time
- cargo llvm-lines
  Lines                  Copies               Function name
  -----                  ------               -------------
  1322611                29544                (TOTAL)
   278720 (21.1%, 21.1%)     1 (0.0%,  0.0%)  turso_core::functions::datetime::parse_modifier

- before:
$> cargo check
warning: `turso_core` (lib) generated 2 warnings
    Finished `dev` profile [unoptimized] target(s) in 5.61s

- after:
$> cargo check
warning: `turso_core` (lib) generated 2 warnings
    Finished `dev` profile [unoptimized] target(s) in 2.24s
2025-11-09 15:15:45 +04:00
Nikita Sivukhin
39daf6df37 ask agent to rewrite compile-heavy function 2025-11-09 15:15:45 +04:00
Pekka Enberg
23be2b2787 Merge 'Stop blob json parsing at null terminator' from Duy Dang
Close #3912

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #3915
2025-11-09 10:03:56 +02:00
Preston Thorpe
43119c1a1e Merge 'Throw an error when adding generated columns via an alter table' from Rohith Suresh
Fixes #3882 and #3887
```bash
turso> create table t(a);
turso> alter table t add column b as (null);
  × Parse error: Alter table does not support adding generated columns
```
```bash
turso> create table t(a); create table s(a);  alter table s add column b references t(a);
turso> select * from sqlite_master;
┌───────┬──────┬──────────┬──────────┬────────────────────────────────────────────────────────┐
│ type  │ name │ tbl_name │ rootpage │ sql                                                    │
├───────┼──────┼──────────┼──────────┼────────────────────────────────────────────────────────┤
│ table │ t    │ t        │        2 │ CREATE TABLE t (a)                                     │
├───────┼──────┼──────────┼──────────┼────────────────────────────────────────────────────────┤
│ table │ s    │ s        │        3 │ CREATE TABLE s (a, b, FOREIGN KEY (b) REFERENCES t(a)) │
└───────┴──────┴──────────┴──────────┴────────────────────────────────────────────────────────┘
turso> 
```

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3884
2025-11-08 09:42:31 -05:00
RS2007
3a562f734c feat: alter table disallow generated columns, support foreign keys for alter table 2025-11-08 13:45:17 +05:30
Pekka Enberg
36c3489c53 github: Make Nyrkiö comment less
We're getting tons of false positives in PRs so let's make Nyrkiö less
verbose.
2025-11-08 10:03:22 +02:00
Pekka Enberg
20364ea2c0 Merge 'core/translate: Remove unused ParamState' from Preston Thorpe
`ParamState` was a leftover from when we were doing some hacky rewriting
of anonymous Parameters/`Expr::Variable`s internally before it was moved
to the parser.

Closes #3926
2025-11-08 10:01:31 +02:00
PThorpe92
dd2e3e8e16 Fix clippy warning 2025-11-07 20:04:57 -05:00
PThorpe92
a012e98bfa core/translate remove unused ParamState and some minor refactoring 2025-11-07 19:18:10 -05:00
pedrocarlo
9007340e99 change convert function to accept 1 value 2025-11-07 12:47:39 -03:00
pedrocarlo
9f350f7fd9 change Text variant in ValueRef to hold a TextRef that can automatically convert to &str avoiding string allocations everywhere 2025-11-07 12:47:39 -03:00
pedrocarlo
5cfc898049 clippy 2025-11-07 12:47:39 -03:00
pedrocarlo
af05d9ba10 move more affinity logic to separate file and avoid more clones 2025-11-07 12:47:39 -03:00
pedrocarlo
61036d5f51 move affinity handling to separate file 2025-11-07 12:47:39 -03:00
pedrocarlo
99c596d340 separate part of comparison logic for reuse later with seek operations 2025-11-07 12:47:39 -03:00
pedrocarlo
ce3527df40 chnage RecordCompare::compare to use an iterator 2025-11-07 12:47:39 -03:00
pedrocarlo
e5e97a5b0a for op_compare reuse compare_immutable 2025-11-07 12:44:57 -03:00
pedrocarlo
b1f4554420 change compare_immutable signature to accept an iterator of AsValueRef 2025-11-07 12:21:59 -03:00
pedrocarlo
9c2324cbd8 move some more functions to be scoped under Value 2025-11-07 12:10:27 -03:00
pedrocarlo
52968b7913 create AsValueRef trait 2025-11-07 12:10:27 -03:00
pedrocarlo
44cab91722 move Value functions to separate file 2025-11-07 12:10:27 -03:00
Pekka Enberg
a6593d109e Merge 'Toy index improvements' from Nikita Sivukhin
This PR implements more sophisticated algorithm in the toy vector sparse
index: now we enumerate components based on the frequency (in order to
check unpopular "features" first) and also estimate length threshold
which can give us better results compared with current top-k set.
Also, this PR adds optional `delta` parameter which can enable
approximate search which will return results with score not more than
`delta` away from the optimal.
In order to implement this index method - index code were slightly
adjusted in order to allow to store some non-key payload in the index
rows. So, now index can hold N columns where first K <= N columns will
be used as identity (before that K always was equal to N).

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3862
2025-11-07 08:29:47 +02:00
Preston Thorpe
4e8b4c96d3 Merge 'use dyn DatabaseStorage instead of DatabaseFile' from Nikita Sivukhin
Partial sync for sync engine will need to implement its own version of
`DatabaseStorage` which willl load database pages on demand

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3922
2025-11-06 15:11:13 -05:00
Preston Thorpe
29c5271c44 Merge 'Prevent DROP TABLE when table is referenced by foreign keys' from Joao Faria
## Related issue
- closes #3885
## Description
Add a check to reject dropping a table when PRAGMA foreign_keys=ON and
the table is referenced by foreign keys

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3913
2025-11-06 15:10:05 -05:00
Preston Thorpe
b037941b01 Merge 'core/vdbe Handle renaming child FK definitions in rename table stmt' from Preston Thorpe
closes https://github.com/tursodatabase/turso/issues/3886

Closes #3904
2025-11-06 13:27:41 -05:00
Nikita Sivukhin
b547f3a25a fix JS tests 2025-11-06 18:02:09 +04:00
Nikita Sivukhin
7294ef347f fix tests 2025-11-06 17:45:12 +04:00
Nikita Sivukhin
da61fa32b4 use dyn DatabaseStorage instead of DatabaseFile 2025-11-06 17:42:03 +04:00
Pekka Enberg
c3d2ea8429 Turso 0.4.0-pre.1 2025-11-06 08:33:13 +02:00
Preston Thorpe
fc78496797 Merge 'Fix EXISTS on LEFT JOIN null rows' from Duy Dang
Close #3890

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3914
2025-11-05 17:06:31 -05:00
Nikita Sivukhin
68a4c90446 fix fmt and test 2025-11-05 20:53:34 +04:00
Nikita Sivukhin
a64aef780d Merge branch 'main' into toy-index-improvements 2025-11-05 20:48:16 +04:00
Nikita Sivukhin
8c449c2ffa rename scratch -> inverted index 2025-11-05 20:47:58 +04:00
Nikita Sivukhin
fe974dd414 fix slice operation implementation 2025-11-05 20:44:43 +04:00
Nikita Sivukhin
fb63a5a3ff final adjustment to align implementation with blog post 2025-11-05 20:44:11 +04:00
Duy Dang
8fba4659c8 Stop blob json parsing at null terminator 2025-11-04 23:04:33 +07:00
joao.faria
2ba643cd68 fix: prevent DROP TABLE when table is referenced by foreign keys
Add foreign key constraint check in translate_drop_table to reject
dropping tables that are referenced by foreign keys when
PRAGMA foreign_keys=ON
2025-11-04 12:32:19 -03:00
Duy Dang
d4b874cc40 Fix EXISTS on LEFT JOIN null rows 2025-11-04 22:01:18 +07:00
PThorpe92
4dffca15da Add some more regression tcl tests for renaming table foreign key checks 2025-11-03 09:47:28 -05:00
PThorpe92
c5a3e590f7 Fix rewriting sql to persist for foreign keys in alter table func 2025-11-03 09:47:28 -05:00
PThorpe92
62d8fd7efd Add regression tcl test for #3886 2025-11-03 09:47:28 -05:00
PThorpe92
ef24911824 Handle renaming child foreign keys on op_rename_table 2025-11-03 09:47:28 -05:00
Pekka Enberg
2c49c47300 Merge 'add some docs for index method' from Nikita Sivukhin
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3909
2025-11-03 15:53:36 +02:00
Nikita Sivukhin
94a39cef3a add some docs for index method 2025-11-03 16:40:18 +04:00
Pekka Enberg
2bf5eb84cf Merge 'Prevent misuse of subqueries that return multiple columns' from Jussi Saurio
Closes #3892
Closes #3888
Stuff like:
```sql
turso>     create table t1(x);
    create table t2(y, z);
    insert into t1 values (1);
    insert into t2 values (1, 2);
    select case (select y, z from t2) when 1 then 'one' else 'other' end from t1;
  × Parse error: base expression in CASE must return 1 value

turso>     create table t(x, y);
    insert into t values (1, 2);
    select (select x, y from t) as result;
  × Parse error: result column must return 1 value, got 2

turso>     create table t1(x,y);
    create table t2(y);
    insert into t1 values (1,1);
    insert into t2 values (1);
    select * from t2 where y = (select x,y from t1);
  × Parse error: all arguments to binary operator = must return the same number of
  │ values. Got: (1) = (2)

turso>     create table orders(customer_id, amount);
    create table thresholds(min_amount, max_amount);
    insert into orders values (100, 50), (100, 150);
    insert into thresholds values (100, 200);
    select customer_id, sum(amount) as total 
    from orders 
    group by customer_id 
    having total > (select min_amount, max_amount from thresholds);
  × Parse error: all arguments to binary operator > must return the same number of
  │ values. Got: (1) > (2)

turso>     create table items(id);
    create table config(max_results, other_col);
    insert into items values (1), (2), (3);
    insert into config values (2, 3);
    select * from items limit (select max_results, other_col from config);
  × Parse error: limit expression must return 1 value, got 2

turso>     create table items(id);
    create table config(skip_count, other_col);
    insert into items values (1), (2), (3);
    insert into config values (1, 2);
    select * from items limit 1 offset (select skip_count, other_col from config);
  × Parse error: offset expression must return 1 value, got 2

turso>     create table items(id, name);
    create table sort_order(priority, other_col);
    insert into items values (1, 'a'), (2, 'b');
    insert into sort_order values (1, 2);
    select * from items order by (select priority, other_col from sort_order);
  × Parse error: order by expression must return 1 value, got 2

turso>     create table sales(product_id, amount);
    create table grouping(category, other_col);
    insert into sales values (1, 100), (2, 200);
    insert into grouping values (1, 2);
    select sum(amount) from sales group by (select category, other_col from grouping);
  × Parse error: group by expression must return 1 value, got 2

turso>     create table t1(x);
    create table t2(y, z);
    insert into t1 values (1);
    insert into t2 values (1, 2);
    select case when (select y, z from t2) then 'yes' else 'no' end from t1;
  × Parse error: when expression in CASE must return 1 value. Got: (2)

turso>     create table t1(x);
    create table t2(y, z);
    insert into t1 values (1);
    insert into t2 values (1, 2);
    select case when x = 1 then (select y, z from t2) else 0 end from t1;
  × Parse error: then expression in CASE must return 1 value. Got: (2)

turso>     create table t1(x);
    create table t2(y, z);
    insert into t1 values (1);
    insert into t2 values (1, 2);
    select case when x = 2 then 0 else (select y, z from t2) end from t1;
  × Parse error: else expression in CASE must return 1 value. Got: (2)

turso>     create table t1(x);
    create table t2(y, z);
    insert into t1 values (1);
    insert into t2 values (1, 2);
    select max((select y, z from t2)) from t1;
  × Parse error: argument 0 to function call max must return 1 value. Got: (2)

turso>     create table t1(x);
    create table t2(y, z);
    insert into t1 values (1);
    insert into t2 values (1, 2);
    select x + (select y, z from t2) from t1;
  × Parse error: all arguments to binary operator + must return the same number of
  │ values. Got: (1) + (2)

turso>     create table t1(x);
    create table t2(y, z);
    insert into t1 values (5);
    insert into t2 values (1, 2);
    select * from t1 where x between (select y, z from t2) and 10;
  × Parse error: all arguments to binary operator <= must return the same number of
  │ values. Got: (2) <= (1)

turso>     create table t1(x);
    create table t2(y, z);
    insert into t1 values (1);
    insert into t2 values (1, 2);
    select cast((select y, z from t2) as integer) from t1;
  × Parse error: argument to CAST must return 1 value. Got: (2)

turso>     create table t1(x);
    create table t2(y, z);
    insert into t1 values (1);
    insert into t2 values ('a', 'b');
    select (select y, z from t2) collate nocase from t1;
  × Parse error: argument to COLLATE must return 1 value. Got: (2)

turso>     create table t1(x);
    create table t2(y, z);
    insert into t1 values (1);
    insert into t2 values (1, 2);
    select * from t1 where (select y, z from t2) is null;
  × Parse error: all arguments to binary operator IS must return the same number of
  │ values. Got: (2) IS (1)

turso>     create table t1(x);
    create table t2(y, z);
    insert into t1 values (1);
    insert into t2 values (1, 2);
    select * from t1 where (select y, z from t2) not null;
  × Parse error: argument to NOT NULL must return 1 value. Got: (2)

turso>     create table t1(x);
    create table t2(y, z);
    insert into t1 values (1);
    insert into t2 values ('a', 'b');
    select * from t1 where (select y, z from t2) like 'a%';
  × Parse error: left operand of LIKE must return 1 value. Got: (2)

turso>     create table t1(x);
    create table t2(y, z);
    insert into t1 values (1);
    insert into t2 values (1, 2);
    select -(select y, z from t2) from t1;
  × Parse error: argument to unary operator - must return 1 value. Got: (2)

turso>     create table t1(x);
    create table t2(y, z);
    insert into t1 values (1);
    insert into t2 values (1, 2);
    select abs((select y, z from t2)) from t1;
  × Parse error: argument 0 to function call abs must return 1 value. Got: (2)
  ```

Closes #3906
2025-11-03 13:06:38 +02:00