Commit Graph

1627 Commits

Author SHA1 Message Date
Nikita Sivukhin
86a95e813d Merge branch 'main' into quoting-fix-attempt-2 2025-09-29 10:58:51 +04:00
Pekka Enberg
d7a0a3db56 Merge 'core/translate: allow creating column called 'rowid'' from Preston Thorpe
closes #3282
includes minor refactor, removing `column_is_rowid_alias`, which is only
checking the public field of the argument Column.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3385
2025-09-27 16:51:09 +03:00
Pekka Enberg
e34a935e0a Merge 'resolve column alias after rewritting column access in the expression in returning insert clause' from Nikita Sivukhin
Fixes https://github.com/tursodatabase/turso/issues/3295

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3355
2025-09-27 16:50:42 +03:00
Glauber Costa
3ee97ddf36 Make sure complex expressions in filters go through Project
We had code for this, but the code had a fatal flaw: it tried to detect
a complex operation (an operation that needs projection), and return
false (no need for projection), for the others.

This is the exact opposite of what we should do: we should identify the
*simple* operations, and then return true (needs projection) for the
rest.

CAST is a special beast, since it is not a function, but rather, a
special opcode. Everything else above is the true just the same. But for
CAST, we have to do the extra work to capture it in the logical plan and
pass it down.

Fixes #3372
Fixes #3370
Fixes #3369
2025-09-27 07:21:03 -03:00
Jussi Saurio
085b92dc4e ALTER TABLE: prevent dropping indexed columns in translate layer 2025-09-27 09:45:15 +03:00
Jussi Saurio
a2d833c073 ALTER TABLE: add comment about things preventing drop column 2025-09-27 09:45:15 +03:00
Pekka Enberg
92291ed736 Merge 'Fix offset variable handling' from Nikita Sivukhin
Before, db generated incorrect plan in case when offset parameter were
introduced as variable:
```
turso> EXPLAIN SELECT * FROM users LIMIT ? OFFSET ?;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     16    0                    0   Start at 16
1     Variable           1     1     0                    0   r[1]=parameter(1); OFFSET expr
2     MustBeInt          1     0     0                    0
3     Variable           2     2     0                    0   r[2]=parameter(2); OFFSET expr
4     MustBeInt          2     0     0                    0
5     OffsetLimit        1     3     2                    0   if r[1]>0 then r[3]=r[1]+max(0,r[2]) else r[3]=(-1)
6     OpenRead           0     2     0                    0   table=users, root=2, iDb=0
7     Rewind             0     15    0                    0   Rewind table users
8       Variable         2     2     0                    0   r[2]=parameter(2); OFFSET expr
9       MustBeInt        2     0     0                    0
10      IfPos            2     14    1                    0   r[2]>0 -> r[2]-=1, goto 14
11      Column           0     0     4                    0   r[4]=users.x
12      ResultRow        4     1     0                    0   output=r[4]
13      DecrJumpZero     1     15    0                    0   if (--r[1]==0) goto 15
14    Next               0     8     0                    0
15    Halt               0     0     0                    0
16    Transaction        0     1     1                    0   iDb=0 tx_mode=Read
17    Goto               0     1     0                    0
```

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3360
2025-09-27 08:20:54 +03:00
PThorpe92
d4dc458328 Evaluate table column refs before checking rowid to allow using it as col name 2025-09-26 17:33:38 -04:00
PThorpe92
af215c2906 Check cols first before falling back to explicit rowid in UPDATE translation 2025-09-26 17:32:16 -04:00
PThorpe92
d9658070a9 Fix clippy warnings 2025-09-26 12:17:34 -04:00
PThorpe92
5fcc187434 translate: refactor arguments and centralize parameter context 2025-09-26 12:06:44 -04:00
Nikita Sivukhin
f80650586a remove misleading comment 2025-09-26 18:06:20 +04:00
Nikita Sivukhin
63a9fa8c28 fix handling of offset parameter set through variable
- before the fix db generated following plan:

turso> EXPLAIN SELECT * FROM users LIMIT ? OFFSET ?;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     16    0                    0   Start at 16
1     Variable           1     1     0                    0   r[1]=parameter(1); OFFSET expr
2     MustBeInt          1     0     0                    0
3     Variable           2     2     0                    0   r[2]=parameter(2); OFFSET expr
4     MustBeInt          2     0     0                    0
5     OffsetLimit        1     3     2                    0   if r[1]>0 then r[3]=r[1]+max(0,r[2]) else r[3]=(-1)
6     OpenRead           0     2     0                    0   table=users, root=2, iDb=0
7     Rewind             0     15    0                    0   Rewind table users
8       Variable         2     2     0                    0   r[2]=parameter(2); OFFSET expr
9       MustBeInt        2     0     0                    0
10      IfPos            2     14    1                    0   r[2]>0 -> r[2]-=1, goto 14
11      Column           0     0     4                    0   r[4]=users.x
12      ResultRow        4     1     0                    0   output=r[4]
13      DecrJumpZero     1     15    0                    0   if (--r[1]==0) goto 15
14    Next               0     8     0                    0
15    Halt               0     0     0                    0
16    Transaction        0     1     1                    0   iDb=0 tx_mode=Read
17    Goto               0     1     0                    0

- the problem here is that Variable value is re-read at step 8 - which is wrong
2025-09-26 18:05:36 +04:00
Nikita Sivukhin
255e357547 resolve column alias after rewritting column access in the expression in returning insert clause 2025-09-26 13:12:46 +04:00
Nikita Sivukhin
f82dd8dffd fix schema sql-gen internal logic to use as_ident() helper 2025-09-26 13:02:35 +04:00
Nikita Sivukhin
27c9506059 do not auto-lowercase Name identifiers
- this is complicated because column names must preserve case
2025-09-26 13:01:49 +04:00
Nikita Sivukhin
c4b3074575 format 2025-09-26 13:01:49 +04:00
Nikita Sivukhin
ae24d637a8 adjust edge-cases 2025-09-26 13:01:49 +04:00
Nikita Sivukhin
12b89fd2f1 do not use Name::new 2025-09-26 13:01:49 +04:00
Nikita Sivukhin
f767a2f8e7 fix compilation 2025-09-26 13:01:49 +04:00
Nikita Sivukhin
f3f9219795 completely remove usage of enum variants 2025-09-26 13:01:49 +04:00
Nikita Sivukhin
9f2f8438e0 remove normalization of identifiers - this will be unnecessary 2025-09-26 13:01:49 +04:00
Nikita Sivukhin
fdf8ca88fd introduce exact(...) function - because enum variant will disappear 2025-09-26 13:01:49 +04:00
Nikita Sivukhin
2f4d76ec6d remove pattern matching over Name::Quoted 2025-09-26 13:01:49 +04:00
Pekka Enberg
506908e648 Merge 'translate: disallow creating/dropping internal tables' from Jussi Saurio
edit: we can't disallow 'turso_' prefix though, because turso-sync-
engine uses it
Closes #3313

Reviewed-by: Nikita Sivukhin (@sivukhin)

Closes #3338
2025-09-26 10:40:09 +03:00
Pekka Enberg
9461e22c06 Merge 'Improve DBSP view serialization' from Glauber Costa
Improve serialization for DBSP views.
The serialization code was written organically, without much forward
thinking about stability as we evolved the table and operator format.
Now that this is done, we are at at point where we can actually make it
suck less and take a considerable step towards making this production
ready.
We also add a simple version check (in the table name, because that is
much easier than reading contents in parse_schema_row) to prevent views
to be used if we had to do anything to evolve the format of the circuit
(including the operators)

Closes #3351
2025-09-26 09:18:45 +03:00
Jussi Saurio
abb0c704af translate: disallow creating/dropping internal tables
we can't disallow 'turso_' prefix though, because turso-sync-engine
uses it
2025-09-26 09:15:32 +03:00
Jussi Saurio
00b69467f3 Merge 'Add CAST to fuzzer' from Levy A.
Adds `CAST` to the fuzzer while fixing some incompatibility bugs.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3314
2025-09-26 09:13:49 +03:00
Jussi Saurio
064cc04b69 Merge 'Fix CREATE INDEX with quoted identifiers' from Iaroslav Zeigerman
Discovered this one while working on #3322
It was a bit more elusive because the original error was essentially a
red herring:
```
turso> CREATE INDEX idx ON "t t" (`a a`);
  × unexpected token at SourceSpan { offset: SourceOffset(22), length: 1 }
   ╭────
 1 │ CREATE INDEX idx ON "t t" (`a a`);
   ·                       ┬
   ·                       ╰── here
   ╰────
  help: expected [TK_LP] but found TK_ID
```

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3345
2025-09-26 09:13:30 +03:00
Jussi Saurio
c83ee9fc3e Merge 'Fix ungrouped aggregate with offset clause' from Preston Thorpe
closes #3300

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3347
2025-09-26 09:13:16 +03:00
Glauber Costa
1b5e74060a make sure that we are able to prevent views from being corrupted
as we make changes to the way materialized views are generated (think
adding new operators, changing the id of existing operators, etc), we
will need to persist the topology of the circuit itself. This is a
change that I believe to be premature. For now, it is enough to reserve
the first operator id for it, and add a version number to the table
name. We can just detect that something changed, and ask the user to
drop the view. We can get away with it due to the fact that the views
are experimental.
2025-09-25 22:52:08 -03:00
Glauber Costa
3dc1dca5a8 use 128-bit hashes for the zset_id
We have used i64 before because that is the size of an integer in
SQLite. However, I believe that for large enough databases, the chances
of collision here are just too high. The effect of a collision is the
database silently returning incorrect data in the materialized view.

So now that everything else is working, we should move to i128.
2025-09-25 22:52:08 -03:00
PThorpe92
3a5b9dcf35 Fix UPDATE behavior to handle explicit rowid update on non rowid alias columns 2025-09-25 19:16:00 -04:00
PThorpe92
429a9e4bf0 Put comment back 2025-09-25 17:14:54 -04:00
PThorpe92
b7a431b79e Fix ungrouped aggregate with offset clause 2025-09-25 16:47:25 -04:00
Iaroslav Zeigerman
b34e53d6c6 Fix CREATE INDEX with quoted identifiers 2025-09-25 12:55:06 -07:00
Pekka Enberg
ad9c12b094 Merge 'Make Connection Send' from Pekka Enberg
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #3292
2025-09-25 21:04:49 +03:00
Pekka Enberg
c255e456d6 Merge 'Fix incorrect "column is ambiguous" error with USING clause' from Jussi Saurio
Closes #3299

Closes #3343
2025-09-25 21:02:36 +03:00
Pekka Enberg
c76738c83b Merge 'Disallow ORDER BY and LIMIT in a non-compound VALUES() statement' from Jussi Saurio
Closes #3280

Closes #3341
2025-09-25 21:02:17 +03:00
Pere Diaz Bou
91cff65e44 Merge 'Autoincrement' from Pavan Nambi
fixes #1976
and #1605
```zsh
turso> DROP TABLE IF EXISTS t;
CREATE TABLE t (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    name TEXT
);
turso> INSERT INTO t (name) VALUES ('A'); SELECT * FROM sqlite_sequence;
┌──────┬─────┐
│ name │ seq │
├──────┼─────┤
│ t    │   1 │
└──────┴─────┘
turso> DROP TABLE IF EXISTS t;
CREATE TABLE t (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    name TEXT
);
turso> INSERT INTO t (name) VALUES ('A'); SELECT * FROM sqlite_sequence;
┌──────┬─────┐
│ name │ seq │
├──────┼─────┤
│ t    │   1 │
└──────┴─────┘
turso> INSERT INTO t (name) VALUES ('A'); SELECT * FROM sqlite_sequence;
┌──────┬─────┐
│ name │ seq │
├──────┼─────┤
│ t    │   2 │
└──────┴─────┘
turso>
```

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2983
2025-09-25 18:57:24 +02:00
Jussi Saurio
433f5a52b1 Disallow ORDER BY and LIMIT in a non-compound VALUES() 2025-09-25 16:30:34 +03:00
Jussi Saurio
00466aa84e Fix incorrect "column is ambiguous" error with USING clause 2025-09-25 16:06:40 +03:00
Pekka Enberg
d8465f5a5e core: Wrap Connection::encryption_key with RwLock 2025-09-25 14:07:39 +03:00
Jussi Saurio
8f32817423 translate: disable support for UPDATE ... ORDER BY
we haven't really tested this at all, and it clearly doesn't work,
as per #3315.

best to disable it for the time being.
2025-09-25 13:54:39 +03:00
Jussi Saurio
c18c44b032 fix: result columns have varying binding precedence
In e.g. `SELECT x AS y, y AS x FROM t ORDER BY x;`, the `x` in the
`ORDER BY` should reference t.y, which has been aliased as `x` for this
query. The same goes for GROUP BY, JOIN ON etc. but NOT for WHERE.

Previously we had wrong precedence in `bind_and_rewrite_expr`.
2025-09-25 08:07:37 +03:00
Levy A.
5dfd67b118 feat: add CAST to fuzzer 2025-09-24 18:06:55 -03:00
Pavan-Nambi
49d5141f2d Merge remote-tracking branch 'origin/main' into cdc_fail_autoincrement 2025-09-24 18:06:02 +05:30
Pekka Enberg
d0e15f9ac0 Merge 'Fix INSERT INTO t DEFAULT VALUES' from Jussi Saurio
Closes #3279

Closes #3291
2025-09-24 11:09:27 +03:00
Jussi Saurio
5c82b72e5f fix INSERT INTO t DEFAULT VALUES 2025-09-24 09:54:43 +03:00
Jussi Saurio
726bc24e78 Support referring to rowid as _rowid_ or oid 2025-09-24 09:17:28 +03:00