Commit Graph

9525 Commits

Author SHA1 Message Date
Pekka Enberg
8d9d2dad1d core/storage: Wrap WalFile::syncing with AtomicBool 2025-09-27 14:07:26 +03:00
Pekka Enberg
9cd869f660 Merge 'Fix various ALTER TABLE bugs' from Jussi Saurio
Closes #3392 , Closes #3395
1. Prevent `ALTER TABLE t DROP COLUMN a` when exists e.g. `CREATE INDEX
ta ON t(a)`
2. Prevent `ALTER TABLE t DROP COLUMN a` when exists e.g. `CREATE INDEX
tba ON t(b) WHERE t.a > 100`
3. Prevent `ALTER TABLE t DROP COLUMN a` when exists e.g. `CREATE VIEW
ta AS SELECT a FROM t`;
and
4. Prevent `ALTER TABLE t RENAME a TO b` when exists e.g. `CREATE VIEW
ta AS SELECT a FROM t`;
Number 4 is incompatible with SQLite because we should be rewriting the
view to include the renamed column. I left that as a FIXME

Closes #3394
2025-09-27 11:10:46 +03:00
Jussi Saurio
283fba2e0d use normalized table name 2025-09-27 09:53:11 +03:00
Jussi Saurio
b43a89e423 Add regression tests for ALTER TABLE stuff 2025-09-27 09:45:15 +03:00
Jussi Saurio
67d320960d ALTER TABLE: prevent dropping/renaming column referenced in VIEW 2025-09-27 09:45:15 +03:00
Jussi Saurio
3137357092 ALTER TABLE: prevent dropping indexed column in VDBE layer 2025-09-27 09:45:15 +03:00
Jussi Saurio
085b92dc4e ALTER TABLE: prevent dropping indexed columns in translate layer 2025-09-27 09:45:15 +03:00
Jussi Saurio
a2d833c073 ALTER TABLE: add comment about things preventing drop column 2025-09-27 09:45:15 +03:00
Pekka Enberg
d796964a1e Merge 'support mixed integer and float expressions in the expr_compiler' from Glauber Costa
Fixes #3373

Closes #3387
2025-09-27 08:22:14 +03:00
Pekka Enberg
e9155610ed Merge 'github: Add 30 minute timeout to all jobs' from Pekka Enberg
We're getting hit by macOS runner concurrency limits whenever some jobs
get stuck (for example, because of a deadlock).

Closes #3377
2025-09-27 08:21:41 +03:00
Pekka Enberg
2f1ac8eb4a Merge 'bench/tpch: remove "cast('yyyy-mm-dd' as datetime)"' from Jussi Saurio
`CAST('yyyy-mm-dd' as datetime)` causes sqlite and tursodb to interpret
the value as just 'yyyy', e.g. '1995-01-01' becomes '1995', causing a
lot of the queries not to return any results, which is not what we want.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3374
2025-09-27 08:21:30 +03:00
Pekka Enberg
65b382b9e1 Merge 'Make MVCC code Send and Sync' from Pekka Enberg
Closes #3361
2025-09-27 08:21:01 +03:00
Pekka Enberg
92291ed736 Merge 'Fix offset variable handling' from Nikita Sivukhin
Before, db generated incorrect plan in case when offset parameter were
introduced as variable:
```
turso> EXPLAIN SELECT * FROM users LIMIT ? OFFSET ?;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     16    0                    0   Start at 16
1     Variable           1     1     0                    0   r[1]=parameter(1); OFFSET expr
2     MustBeInt          1     0     0                    0
3     Variable           2     2     0                    0   r[2]=parameter(2); OFFSET expr
4     MustBeInt          2     0     0                    0
5     OffsetLimit        1     3     2                    0   if r[1]>0 then r[3]=r[1]+max(0,r[2]) else r[3]=(-1)
6     OpenRead           0     2     0                    0   table=users, root=2, iDb=0
7     Rewind             0     15    0                    0   Rewind table users
8       Variable         2     2     0                    0   r[2]=parameter(2); OFFSET expr
9       MustBeInt        2     0     0                    0
10      IfPos            2     14    1                    0   r[2]>0 -> r[2]-=1, goto 14
11      Column           0     0     4                    0   r[4]=users.x
12      ResultRow        4     1     0                    0   output=r[4]
13      DecrJumpZero     1     15    0                    0   if (--r[1]==0) goto 15
14    Next               0     8     0                    0
15    Halt               0     0     0                    0
16    Transaction        0     1     1                    0   iDb=0 tx_mode=Read
17    Goto               0     1     0                    0
```

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3360
2025-09-27 08:20:54 +03:00
Pekka Enberg
a932ac5450 Merge 'core: recover logical log on Database::connect ' from Pere Diaz Bou
If we open database and logical log is not empty we need to recover from
it. We also make sure a single recover executes concurrently and other
connections just wait for it to finish.
I also changed the fuzz tester to use `restart` instead of calling
`load_logical_log` manually to test this behaviour.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3359
2025-09-27 08:20:41 +03:00
Glauber Costa
d28022b491 support mixed integer and float expressions in the expr_compiler
Fixes #3373
2025-09-26 21:11:38 -03:00
Preston Thorpe
5dc30b5a68 Merge 'core/storage: Display page category for rowid integrity check failure' from Pekka Enberg
Let's add more hints to hunt down the reason for #2896.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3378
2025-09-26 19:07:45 -04:00
Preston Thorpe
ddbedecace Merge 'fix encryption config in the sync-client' from Nikita Sivukhin
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3379
2025-09-26 19:01:27 -04:00
Preston Thorpe
e43184fb98 Merge 'translate: refactor arguments and centralize parameter context' from Preston Thorpe
This PR contains _no semantic changes_.
I made this cleanup on another branch when I was working on subqueries,
as the inconsistency with passing around `Schema` and `SymbolTable` had
been kinda bothering me. By adding `param_ctx` to the program, it
prevents accidentally resetting it to zero.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3382
2025-09-26 13:05:29 -04:00
PThorpe92
d9658070a9 Fix clippy warnings 2025-09-26 12:17:34 -04:00
PThorpe92
5fcc187434 translate: refactor arguments and centralize parameter context 2025-09-26 12:06:44 -04:00
Pekka Enberg
222ab125c1 Turso 0.2.0-pre.9 2025-09-26 19:00:14 +03:00
Nikita Sivukhin
fe4bfb7c88 fix encryption config in the sync-client 2025-09-26 19:29:37 +04:00
Pekka Enberg
931cf2658e core/storage: Display page category for rowid integrity check failure
Let's add more hints to hunt down the reason for #2896.
2025-09-26 18:25:49 +03:00
Pere Diaz Bou
99adf73168 core/mvcc/logical-log: rename to needs_recovery 2025-09-26 16:59:57 +02:00
Pekka Enberg
f7bf60e856 github: Add 30 minute timeout to all jobs
We're getting hit by macOS runner concurrency limits whenever some jobs
get stuck (for example, because of a deadlock).
2025-09-26 17:45:42 +03:00
Nikita Sivukhin
f80650586a remove misleading comment 2025-09-26 18:06:20 +04:00
Nikita Sivukhin
5bf69350b3 add simple tests for offset/limit binding 2025-09-26 18:06:20 +04:00
Nikita Sivukhin
63a9fa8c28 fix handling of offset parameter set through variable
- before the fix db generated following plan:

turso> EXPLAIN SELECT * FROM users LIMIT ? OFFSET ?;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     16    0                    0   Start at 16
1     Variable           1     1     0                    0   r[1]=parameter(1); OFFSET expr
2     MustBeInt          1     0     0                    0
3     Variable           2     2     0                    0   r[2]=parameter(2); OFFSET expr
4     MustBeInt          2     0     0                    0
5     OffsetLimit        1     3     2                    0   if r[1]>0 then r[3]=r[1]+max(0,r[2]) else r[3]=(-1)
6     OpenRead           0     2     0                    0   table=users, root=2, iDb=0
7     Rewind             0     15    0                    0   Rewind table users
8       Variable         2     2     0                    0   r[2]=parameter(2); OFFSET expr
9       MustBeInt        2     0     0                    0
10      IfPos            2     14    1                    0   r[2]>0 -> r[2]-=1, goto 14
11      Column           0     0     4                    0   r[4]=users.x
12      ResultRow        4     1     0                    0   output=r[4]
13      DecrJumpZero     1     15    0                    0   if (--r[1]==0) goto 15
14    Next               0     8     0                    0
15    Halt               0     0     0                    0
16    Transaction        0     1     1                    0   iDb=0 tx_mode=Read
17    Goto               0     1     0                    0

- the problem here is that Variable value is re-read at step 8 - which is wrong
2025-09-26 18:05:36 +04:00
Preston Thorpe
1b2d7ea534 Merge 'fix avg aggregation' from Nikita Sivukhin
- ignore NULL rows as SQLite do
- emit NULL instead of NaN when no rows were aggregated
- adjust agg column alias name

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3376
2025-09-26 09:59:50 -04:00
Nikita Sivukhin
a0c47b98b8 fix test 2025-09-26 17:42:11 +04:00
Nikita Sivukhin
5b5379d078 propagate context to stringifier to properly derive column names 2025-09-26 17:40:41 +04:00
Nikita Sivukhin
52f3216211 fix avg aggregation
- ignore NULL rows as SQLite do
- emit NULL instead of NaN when no rows were aggregated
- adjust agg column alias name
2025-09-26 17:11:06 +04:00
Jussi Saurio
045b11b255 bench/tpc-h: don't fail build if query 1 has output difference (known floating point precision issue) 2025-09-26 16:02:37 +03:00
Jussi Saurio
a783f82470 bench/tpch: remove "cast('yyyy-mm-dd' as datetime)"
this causes sqlite and tursodb to interpret the value as just 'yyyy',
e.g. '1995-01-01' becomes '1995', causing a lot of the queries not to
return any results, which is not what we want.
2025-09-26 15:55:38 +03:00
Pere Diaz Bou
9e47cc3700 clippy 2025-09-26 14:16:11 +02:00
Pekka Enberg
1402e9841e core/mvcc: Wrap StreamingLogicalLogReader::buffer with RwLock 2025-09-26 14:19:16 +03:00
Pekka Enberg
96accef06c core/mvcc: Wrap header with RwLock 2025-09-26 14:10:18 +03:00
Pere Diaz Bou
59d3e37b9f fmt 2025-09-26 13:01:13 +02:00
Pere Diaz Bou
9c1d94a355 core/mvcc/logical-log: assert we don't call begin_load_tx twice 2025-09-26 12:59:13 +02:00
Pere Diaz Bou
4cdf293a2b core/mvcc/logical-log: fuzz test recover use db.restart 2025-09-26 12:56:58 +02:00
Pere Diaz Bou
ae994146af core/mvcc/logical-log: on mvcc restart clear DATABASE_MANAGER 2025-09-26 12:56:43 +02:00
Pere Diaz Bou
2a7abd82f7 core/lib: recover mvcc logical log if needed on connect 2025-09-26 12:47:52 +02:00
Pere Diaz Bou
83d8a7c775 core/mvcc/logical-log: lock recover logical log process 2025-09-26 12:47:31 +02:00
Pekka Enberg
506908e648 Merge 'translate: disallow creating/dropping internal tables' from Jussi Saurio
edit: we can't disallow 'turso_' prefix though, because turso-sync-
engine uses it
Closes #3313

Reviewed-by: Nikita Sivukhin (@sivukhin)

Closes #3338
2025-09-26 10:40:09 +03:00
Pekka Enberg
9461e22c06 Merge 'Improve DBSP view serialization' from Glauber Costa
Improve serialization for DBSP views.
The serialization code was written organically, without much forward
thinking about stability as we evolved the table and operator format.
Now that this is done, we are at at point where we can actually make it
suck less and take a considerable step towards making this production
ready.
We also add a simple version check (in the table name, because that is
much easier than reading contents in parse_schema_row) to prevent views
to be used if we had to do anything to evolve the format of the circuit
(including the operators)

Closes #3351
2025-09-26 09:18:45 +03:00
Jussi Saurio
cfa449a0c0 Merge 'Disallow multiple primary keys in table definition' from Jussi Saurio
Closes #3309

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #3340
2025-09-26 09:16:14 +03:00
Jussi Saurio
abb0c704af translate: disallow creating/dropping internal tables
we can't disallow 'turso_' prefix though, because turso-sync-engine
uses it
2025-09-26 09:15:32 +03:00
Jussi Saurio
00b69467f3 Merge 'Add CAST to fuzzer' from Levy A.
Adds `CAST` to the fuzzer while fixing some incompatibility bugs.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3314
2025-09-26 09:13:49 +03:00
Jussi Saurio
064cc04b69 Merge 'Fix CREATE INDEX with quoted identifiers' from Iaroslav Zeigerman
Discovered this one while working on #3322
It was a bit more elusive because the original error was essentially a
red herring:
```
turso> CREATE INDEX idx ON "t t" (`a a`);
  × unexpected token at SourceSpan { offset: SourceOffset(22), length: 1 }
   ╭────
 1 │ CREATE INDEX idx ON "t t" (`a a`);
   ·                       ┬
   ·                       ╰── here
   ╰────
  help: expected [TK_LP] but found TK_ID
```

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3345
2025-09-26 09:13:30 +03:00
Jussi Saurio
c83ee9fc3e Merge 'Fix ungrouped aggregate with offset clause' from Preston Thorpe
closes #3300

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3347
2025-09-26 09:13:16 +03:00