Commit Graph

6027 Commits

Author SHA1 Message Date
Jussi Saurio
b024fdb17d Merge 'core: update aegis' from Daeho Ro
It seems that the build on macos arm is failing with `aegis` v0.9.0.
So, here I update `aegis`.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3561
2025-11-10 11:27:01 +02:00
Pekka Enberg
b74ddf30f9 Merge 'extensions/vtabs: implement remaining opcodes' from Preston Thorpe
The only real benefit right now here is the ability to rename virtual
tables.
Then this now properly calls `VBegin` at the start of a vtab write
transaction, despite none of our extensions needing or implementing
transactions at this point.
```console
explain insert into t values ('key','value');
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     10    0                    0   Start at 10
1     VOpen              0     0     0                    0    t
2     VBegin             0     0     0                    0   
3     Null               0     1     0                    0   r[1]=NULL
4     Null               0     3     0                    0   r[3]=NULL
5     String8            0     4     0     key            0   r[4]='key'
6     String8            0     5     0     value          0   r[5]='value'
7     VUpdate            0     5     1                    0   args=r[1..5]
8     Close              0     0     0                    0   
9     Halt               0     0     0                    0   
10    Transaction        0     2     1                    0   iDb=0 tx_mode=Write
11    Goto               0     1     0                    0   
Exiting Turso SQL Shell.
```

Closes #3930
2025-11-10 09:03:07 +02:00
Pekka Enberg
7891be96fd Merge 'Refactor affinity conversions for reusability' from Pedro Muniz
Depends on #3920
Moves some code around so it is easier to reuse and less cluttered in
`execute.rs`, and changes how `compare` works. Instead of mutating some
register, we now just return the possible `ValueRef` representation of
that affinity. This allows other parts of the codebase to reuse this
logic without needing to have an owned `Value` or a `&mut Register`

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3923
2025-11-10 09:02:22 +02:00
Pekka Enberg
2be515247f Merge 'Create AsValueRef trait to allow us to be agnostic over ownership of Value or ValueRef' from Pedro Muniz
Depends on #3919
Also change `op_compare` to reuse the same compare_immutable logic
First step to finish #2304

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3920
2025-11-10 09:01:59 +02:00
Pekka Enberg
4bb0edac5e Merge 'Move value functions to separate file' from Pedro Muniz
Makes it easier to visualize what is related to Value and what is
related to opcodes. This will also facilitate in my next PR to
generalize certain function over `Value` and `ValueRef` as listed in
#2304

Closes #3919
2025-11-10 09:01:29 +02:00
PThorpe92
5c207618a7 Fix extensions py test 2025-11-09 11:35:57 -05:00
PThorpe92
b443b09516 Remove VRollback and VCommit as they are unused opcodes in sqlite 2025-11-09 11:27:09 -05:00
PThorpe92
94b6d254a9 Fix comment on vtab_txn_states 2025-11-09 11:08:52 -05:00
PThorpe92
993c9d34b4 Rollback vtab txns when when err code is present in Halt 2025-11-09 11:07:43 -05:00
PThorpe92
f35ccfba17 Add support for renaming virtual tables 2025-11-09 11:07:42 -05:00
PThorpe92
e09d9eb720 Add VBegin, VRename, VRollback and VCommit opcodes 2025-11-09 11:07:42 -05:00
Nikita Sivukhin
6e0e4e20ab add comment 2025-11-09 16:04:16 +04:00
Nikita Sivukhin
ec91829835 rewrite parse_modifier function because its current version lead to enormous amount of generated LLVM code which significantly increase compilation time
- cargo llvm-lines
  Lines                  Copies               Function name
  -----                  ------               -------------
  1322611                29544                (TOTAL)
   278720 (21.1%, 21.1%)     1 (0.0%,  0.0%)  turso_core::functions::datetime::parse_modifier

- before:
$> cargo check
warning: `turso_core` (lib) generated 2 warnings
    Finished `dev` profile [unoptimized] target(s) in 5.61s

- after:
$> cargo check
warning: `turso_core` (lib) generated 2 warnings
    Finished `dev` profile [unoptimized] target(s) in 2.24s
2025-11-09 15:15:45 +04:00
Nikita Sivukhin
39daf6df37 ask agent to rewrite compile-heavy function 2025-11-09 15:15:45 +04:00
Pekka Enberg
23be2b2787 Merge 'Stop blob json parsing at null terminator' from Duy Dang
Close #3912

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #3915
2025-11-09 10:03:56 +02:00
RS2007
3a562f734c feat: alter table disallow generated columns, support foreign keys for alter table 2025-11-08 13:45:17 +05:30
PThorpe92
dd2e3e8e16 Fix clippy warning 2025-11-07 20:04:57 -05:00
PThorpe92
a012e98bfa core/translate remove unused ParamState and some minor refactoring 2025-11-07 19:18:10 -05:00
pedrocarlo
9007340e99 change convert function to accept 1 value 2025-11-07 12:47:39 -03:00
pedrocarlo
9f350f7fd9 change Text variant in ValueRef to hold a TextRef that can automatically convert to &str avoiding string allocations everywhere 2025-11-07 12:47:39 -03:00
pedrocarlo
5cfc898049 clippy 2025-11-07 12:47:39 -03:00
pedrocarlo
af05d9ba10 move more affinity logic to separate file and avoid more clones 2025-11-07 12:47:39 -03:00
pedrocarlo
61036d5f51 move affinity handling to separate file 2025-11-07 12:47:39 -03:00
pedrocarlo
99c596d340 separate part of comparison logic for reuse later with seek operations 2025-11-07 12:47:39 -03:00
pedrocarlo
ce3527df40 chnage RecordCompare::compare to use an iterator 2025-11-07 12:47:39 -03:00
pedrocarlo
e5e97a5b0a for op_compare reuse compare_immutable 2025-11-07 12:44:57 -03:00
pedrocarlo
b1f4554420 change compare_immutable signature to accept an iterator of AsValueRef 2025-11-07 12:21:59 -03:00
pedrocarlo
9c2324cbd8 move some more functions to be scoped under Value 2025-11-07 12:10:27 -03:00
pedrocarlo
52968b7913 create AsValueRef trait 2025-11-07 12:10:27 -03:00
pedrocarlo
44cab91722 move Value functions to separate file 2025-11-07 12:10:27 -03:00
Pekka Enberg
a6593d109e Merge 'Toy index improvements' from Nikita Sivukhin
This PR implements more sophisticated algorithm in the toy vector sparse
index: now we enumerate components based on the frequency (in order to
check unpopular "features" first) and also estimate length threshold
which can give us better results compared with current top-k set.
Also, this PR adds optional `delta` parameter which can enable
approximate search which will return results with score not more than
`delta` away from the optimal.
In order to implement this index method - index code were slightly
adjusted in order to allow to store some non-key payload in the index
rows. So, now index can hold N columns where first K <= N columns will
be used as identity (before that K always was equal to N).

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3862
2025-11-07 08:29:47 +02:00
Preston Thorpe
4e8b4c96d3 Merge 'use dyn DatabaseStorage instead of DatabaseFile' from Nikita Sivukhin
Partial sync for sync engine will need to implement its own version of
`DatabaseStorage` which willl load database pages on demand

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3922
2025-11-06 15:11:13 -05:00
Preston Thorpe
29c5271c44 Merge 'Prevent DROP TABLE when table is referenced by foreign keys' from Joao Faria
## Related issue
- closes #3885
## Description
Add a check to reject dropping a table when PRAGMA foreign_keys=ON and
the table is referenced by foreign keys

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3913
2025-11-06 15:10:05 -05:00
Preston Thorpe
b037941b01 Merge 'core/vdbe Handle renaming child FK definitions in rename table stmt' from Preston Thorpe
closes https://github.com/tursodatabase/turso/issues/3886

Closes #3904
2025-11-06 13:27:41 -05:00
Nikita Sivukhin
7294ef347f fix tests 2025-11-06 17:45:12 +04:00
Nikita Sivukhin
da61fa32b4 use dyn DatabaseStorage instead of DatabaseFile 2025-11-06 17:42:03 +04:00
Nikita Sivukhin
68a4c90446 fix fmt and test 2025-11-05 20:53:34 +04:00
Nikita Sivukhin
a64aef780d Merge branch 'main' into toy-index-improvements 2025-11-05 20:48:16 +04:00
Nikita Sivukhin
8c449c2ffa rename scratch -> inverted index 2025-11-05 20:47:58 +04:00
Nikita Sivukhin
fe974dd414 fix slice operation implementation 2025-11-05 20:44:43 +04:00
Nikita Sivukhin
fb63a5a3ff final adjustment to align implementation with blog post 2025-11-05 20:44:11 +04:00
Duy Dang
8fba4659c8 Stop blob json parsing at null terminator 2025-11-04 23:04:33 +07:00
joao.faria
2ba643cd68 fix: prevent DROP TABLE when table is referenced by foreign keys
Add foreign key constraint check in translate_drop_table to reject
dropping tables that are referenced by foreign keys when
PRAGMA foreign_keys=ON
2025-11-04 12:32:19 -03:00
Duy Dang
d4b874cc40 Fix EXISTS on LEFT JOIN null rows 2025-11-04 22:01:18 +07:00
PThorpe92
c5a3e590f7 Fix rewriting sql to persist for foreign keys in alter table func 2025-11-03 09:47:28 -05:00
PThorpe92
ef24911824 Handle renaming child foreign keys on op_rename_table 2025-11-03 09:47:28 -05:00
Pekka Enberg
2bf5eb84cf Merge 'Prevent misuse of subqueries that return multiple columns' from Jussi Saurio
Closes #3892
Closes #3888
Stuff like:
```sql
turso>     create table t1(x);
    create table t2(y, z);
    insert into t1 values (1);
    insert into t2 values (1, 2);
    select case (select y, z from t2) when 1 then 'one' else 'other' end from t1;
  × Parse error: base expression in CASE must return 1 value

turso>     create table t(x, y);
    insert into t values (1, 2);
    select (select x, y from t) as result;
  × Parse error: result column must return 1 value, got 2

turso>     create table t1(x,y);
    create table t2(y);
    insert into t1 values (1,1);
    insert into t2 values (1);
    select * from t2 where y = (select x,y from t1);
  × Parse error: all arguments to binary operator = must return the same number of
  │ values. Got: (1) = (2)

turso>     create table orders(customer_id, amount);
    create table thresholds(min_amount, max_amount);
    insert into orders values (100, 50), (100, 150);
    insert into thresholds values (100, 200);
    select customer_id, sum(amount) as total 
    from orders 
    group by customer_id 
    having total > (select min_amount, max_amount from thresholds);
  × Parse error: all arguments to binary operator > must return the same number of
  │ values. Got: (1) > (2)

turso>     create table items(id);
    create table config(max_results, other_col);
    insert into items values (1), (2), (3);
    insert into config values (2, 3);
    select * from items limit (select max_results, other_col from config);
  × Parse error: limit expression must return 1 value, got 2

turso>     create table items(id);
    create table config(skip_count, other_col);
    insert into items values (1), (2), (3);
    insert into config values (1, 2);
    select * from items limit 1 offset (select skip_count, other_col from config);
  × Parse error: offset expression must return 1 value, got 2

turso>     create table items(id, name);
    create table sort_order(priority, other_col);
    insert into items values (1, 'a'), (2, 'b');
    insert into sort_order values (1, 2);
    select * from items order by (select priority, other_col from sort_order);
  × Parse error: order by expression must return 1 value, got 2

turso>     create table sales(product_id, amount);
    create table grouping(category, other_col);
    insert into sales values (1, 100), (2, 200);
    insert into grouping values (1, 2);
    select sum(amount) from sales group by (select category, other_col from grouping);
  × Parse error: group by expression must return 1 value, got 2

turso>     create table t1(x);
    create table t2(y, z);
    insert into t1 values (1);
    insert into t2 values (1, 2);
    select case when (select y, z from t2) then 'yes' else 'no' end from t1;
  × Parse error: when expression in CASE must return 1 value. Got: (2)

turso>     create table t1(x);
    create table t2(y, z);
    insert into t1 values (1);
    insert into t2 values (1, 2);
    select case when x = 1 then (select y, z from t2) else 0 end from t1;
  × Parse error: then expression in CASE must return 1 value. Got: (2)

turso>     create table t1(x);
    create table t2(y, z);
    insert into t1 values (1);
    insert into t2 values (1, 2);
    select case when x = 2 then 0 else (select y, z from t2) end from t1;
  × Parse error: else expression in CASE must return 1 value. Got: (2)

turso>     create table t1(x);
    create table t2(y, z);
    insert into t1 values (1);
    insert into t2 values (1, 2);
    select max((select y, z from t2)) from t1;
  × Parse error: argument 0 to function call max must return 1 value. Got: (2)

turso>     create table t1(x);
    create table t2(y, z);
    insert into t1 values (1);
    insert into t2 values (1, 2);
    select x + (select y, z from t2) from t1;
  × Parse error: all arguments to binary operator + must return the same number of
  │ values. Got: (1) + (2)

turso>     create table t1(x);
    create table t2(y, z);
    insert into t1 values (5);
    insert into t2 values (1, 2);
    select * from t1 where x between (select y, z from t2) and 10;
  × Parse error: all arguments to binary operator <= must return the same number of
  │ values. Got: (2) <= (1)

turso>     create table t1(x);
    create table t2(y, z);
    insert into t1 values (1);
    insert into t2 values (1, 2);
    select cast((select y, z from t2) as integer) from t1;
  × Parse error: argument to CAST must return 1 value. Got: (2)

turso>     create table t1(x);
    create table t2(y, z);
    insert into t1 values (1);
    insert into t2 values ('a', 'b');
    select (select y, z from t2) collate nocase from t1;
  × Parse error: argument to COLLATE must return 1 value. Got: (2)

turso>     create table t1(x);
    create table t2(y, z);
    insert into t1 values (1);
    insert into t2 values (1, 2);
    select * from t1 where (select y, z from t2) is null;
  × Parse error: all arguments to binary operator IS must return the same number of
  │ values. Got: (2) IS (1)

turso>     create table t1(x);
    create table t2(y, z);
    insert into t1 values (1);
    insert into t2 values (1, 2);
    select * from t1 where (select y, z from t2) not null;
  × Parse error: argument to NOT NULL must return 1 value. Got: (2)

turso>     create table t1(x);
    create table t2(y, z);
    insert into t1 values (1);
    insert into t2 values ('a', 'b');
    select * from t1 where (select y, z from t2) like 'a%';
  × Parse error: left operand of LIKE must return 1 value. Got: (2)

turso>     create table t1(x);
    create table t2(y, z);
    insert into t1 values (1);
    insert into t2 values (1, 2);
    select -(select y, z from t2) from t1;
  × Parse error: argument to unary operator - must return 1 value. Got: (2)

turso>     create table t1(x);
    create table t2(y, z);
    insert into t1 values (1);
    insert into t2 values (1, 2);
    select abs((select y, z from t2)) from t1;
  × Parse error: argument 0 to function call abs must return 1 value. Got: (2)
  ```

Closes #3906
2025-11-03 13:06:38 +02:00
Pekka Enberg
9aae220509 Merge 'Optimize and refactor schema::Column type' from Preston Thorpe
closes https://github.com/tursodatabase/turso/issues/3773
## Before
```rust
#[derive(Debug, Clone)]
pub struct Column {
    pub name: Option<String>,
    pub ty: Type,
    // many sqlite operations like table_info retain the original string
    pub ty_str: String,
    pub primary_key: bool,
    pub is_rowid_alias: bool,
    pub notnull: bool,
    pub default: Option<Box<Expr>>,
    pub unique: bool,
    pub collation: Option<CollationSeq>,
    pub hidden: bool,
}
```
obviously not ideal. so lets pack `type`, `hidden`, `primary_key`,
`is_rowid_alias`, `notnull` and `collation` into a u16.
## After:
```rust
#[derive(Debug, Clone)]
pub struct Column {
    pub name: Option<String>,
    pub ty_str: String,
    pub default: Option<Box<Expr>>,
    raw: u16,
}
```
Also saw a place to replace a `Mutex<Enum>` with `AtomicEnum`, so I
snuck that in here too

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3905
2025-11-03 13:05:35 +02:00
Jussi Saurio
1c2a8e62ca Fix: return error on provided insert column count mismatch 2025-11-03 11:41:50 +02:00
Jussi Saurio
005d922ab4 Fix: prevent misuse of subqueries that return multiple columns 2025-11-03 11:04:09 +02:00