Commit Graph

4980 Commits

Author SHA1 Message Date
Pekka Enberg
84cf3640cb Merge 'Enable encryption option in Whopper ' from Avinash Sajjanshetty
If encryption is arg is passed, then we run whopper tests with
encryptions. We randomly generate cipher and key, and run whopper. They
are also printed for debugging and analysis.
Also, updated the corresponding scripts. So now you can do:
```bash
./whopper/bin/run --enable-encryption

or 

./whopper/bin/explore --enable-encryption
```

Closes #3183
2025-09-18 10:03:13 +03:00
Jussi Saurio
1d2b461a2a Merge 'Compat: Translate the 2nd argument of group_concat / string_agg' from Iaroslav Zeigerman
Fixes #3140

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3155
2025-09-18 09:23:05 +03:00
Jussi Saurio
0d37ac2519 Merge 'translate: couple fixes from testing with Gorm' from Preston Thorpe
Ongoing tests for [turso-go](https://github.com/tursodatabase/turso-go)
have unearthed a couple more issues
closes #3187
### Number 1:
We were getting something like:
```sql
sqlite_autoindex_`databases`_2
```
when creating autoindex for table in Gorm (gorm is notorious for
backticks everywhere), because of not normalizing the column name when
creating autoindex.
### Number 2:
When creating table with `PRIMARY KEY AUTOINCREMENT`, we were still
creating the index, but it wasn't properly handled in
`populate_indices`, because we are doing the following:
```rust
                if column.primary_key && unique_set.is_primary_key {
                    if column.is_rowid_alias {
                        // rowid alias, no index needed
                        continue; // continues, but doesn't consume it..
                    }
```
So if we created such an index entry for the AUTOINCREMENT... we would
trip this:
```rust
assert!(automatic_indexes.is_empty(), "all automatic indexes parsed from sqlite_schema should have been consumed, but {} remain", automatic_indexes.len());
```

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3186
2025-09-18 09:21:41 +03:00
Jussi Saurio
498293658e Merge 'Reduce allocations needed for break_predicate_at_and_boundaries' from Lâm Hoàng Phúc
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3188
2025-09-18 09:21:24 +03:00
Jussi Saurio
91ef4e5e9d Merge 'Introduce instruction VTABLE' from Lâm Hoàng Phúc
this PR improves 3-6% for `prepare` benchmark without slowing down
others.  After this PR we don't have to store `InsnFunction`  in
`Program` and `ProgramBuilder` anymore, because `to_function` will
return result without matching.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3098
2025-09-18 09:18:48 +03:00
Pekka Enberg
d2376a239a Merge 'core/mvcc: introduce with_header for MVCC header update tracking' from Pere Diaz Bou
Currently header changes are tracked through pager by reading page 1.
MVCC has it's own layer to track changes during txn so this commit makes
it so that headers are tracked by each txn separately.
On commit we update the _global_ header which is used to update
`database_size` because pager commits require it to be up to date. This
also makes it _simpler_ to keep track of header updates and update
pager's header accordingly.
This PR is needed in order to make logical log work because we don't
want to rely on pager as much as possible!

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3156
2025-09-18 08:13:14 +03:00
TcMits
88119888d0 reduce allocation needed for break_predicate_at_and_boundaries 2025-09-18 10:52:29 +07:00
PThorpe92
5aa07eb826 Use normalized table name for autoindex creation 2025-09-17 20:51:22 -04:00
PThorpe92
45fcadbf20 Fix formatting of autoindex 2025-09-17 16:07:11 -04:00
PThorpe92
cfc6ec4ee5 Normalize more idents 2025-09-17 15:53:06 -04:00
PThorpe92
c57545d504 Avoid panicking when we create autoindex for AUTOINCREMENT primary key 2025-09-17 15:52:42 -04:00
PThorpe92
dde8a49f4e normalize identifier for creating autoindex to prevent e.g. sqlite_autoindextable_2 2025-09-17 13:25:33 -04:00
PThorpe92
4e71524e42 normalize identifier for ID::Name in upsert expr rewriting 2025-09-17 13:24:06 -04:00
Avinash Sajjanshetty
f9a6cde79a Make encryption::CipherMode public 2025-09-17 21:55:23 +05:30
Preston Thorpe
8c53d7f024 Merge 'translation: rewrite expressions and properly handle quoted identifiers in UPSERT' from Preston Thorpe
This PR fixes bugs found in the [turso-
go](https://github.com/tursodatabase/turso-go) driver with UPSERT clause
earlier, where `Gorm` will (obviously) use Expr::Variable's as well as
use quotes for `Expr::Qualified` in the tail end of an UPSERT statement.
Example:
```sql
INSERT INTO users (a,b,c) VALUES (?,?,?) ON CONFLICT (`users`.`a`) DO UPDATE SET b = `excluded`.`b`, a = ?;
```
and previously we were not properly calling `rewrite_expr`, which was
not properly setting the anonymous `Expr::Variable` to `__param_N` named
parameter, so it would ignore it completely, then return the wrong # of
parameters.
Also, we didn't handle quoted "`excluded`.`x`", so it would panic in the
optimizer that Qualified should have been rewritten earlier.

Closes #3157
2025-09-17 11:25:13 -04:00
Iaroslav Zeigerman
29e0cabf2a Compat: Translate the 2nd argument of group_concat / string_agg 2025-09-17 07:42:07 -07:00
Jussi Saurio
8bf52de94b Merge 'Remove serialization of normal write/commit path' from Preston Thorpe
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3089
2025-09-17 17:30:45 +03:00
Pekka Enberg
f9db3e97fd Merge 'core/vtab: Wrap InternalVirtualTable with RwLock' from Pekka Enberg
Closes #3180
2025-09-17 16:40:20 +03:00
Pekka Enberg
617adf8eba Merge 'Clean up encryption feature flag usage' from Avinash Sajjanshetty
Closes #3178
2025-09-17 16:40:08 +03:00
Pekka Enberg
5ebf6cd128 core/vtab: Wrap InternalVirtualTable with RwLock 2025-09-17 15:43:33 +03:00
Pekka Enberg
b98e1efb17 Merge 'core/storage: Wrap Pager::checkpoint_state in RwLock' from Pekka Enberg
Closes #3176
2025-09-17 14:41:18 +03:00
Avinash Sajjanshetty
4eaee0fd81 clean up encryption feature flag usage 2025-09-17 16:30:51 +05:30
Preston Thorpe
bcafb288ad Merge 'Fix is_nonnull returns true on 1 / 0' from Lâm Hoàng Phúc
turso:
```sh
turso> CREATE TABLE t (x PRIMARY KEY, y, z);
turso> INSERT INTO t VALUES (37, -70, -196792117);
turso> SELECT * FROM t WHERE  (1 / 0) >= -3289742039 < t.x;
┌────┬─────┬────────────┐
│ x  │ y   │ z          │
├────┼─────┼────────────┤
│ 37 │ -70 │ -196792117 │
└────┴─────┴────────────┘
turso>
```
sqlite:
```sh
sqlite> CREATE TABLE t (x PRIMARY KEY, y, z);
sqlite> INSERT INTO t VALUES (37, -70, -196792117);
sqlite> SELECT * FROM t WHERE  (1 / 0) >= -3289742039 < t.x;
sqlite>
```
related: https://github.com/tursodatabase/turso/actions/runs/17765571409
/job/50488042583?pr=3147#step:8:855

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #3167
2025-09-17 06:55:55 -04:00
PThorpe92
97c11898fe Minor refactor in translate/insert 2025-09-17 06:44:10 -04:00
PThorpe92
5dd466941e Handle upsert even in inserting_multiple_rows case 2025-09-17 06:44:09 -04:00
PThorpe92
85eee42bf1 Support quoted qualified identifiers in UPSERT excluded.x clauses 2025-09-17 06:44:08 -04:00
PThorpe92
d2cd833b86 Rewrite exprs in set + where clause for UPSERT 2025-09-17 06:38:25 -04:00
Pekka Enberg
2b89dd4ea8 Merge 'core: Wrap Pager dirty_pages in RwLock' from Pekka Enberg
Make it Sync and Send.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3175
2025-09-17 13:34:47 +03:00
Pekka Enberg
58b0e8665d Merge 'core: Wrap MvCursor in Arc<RwLock<>>' from Pekka Enberg
Make it Send and Sync.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3174
2025-09-17 13:34:34 +03:00
Pere Diaz Bou
434daf4b10 core/mvcc: fmt 2025-09-17 12:18:44 +02:00
Pere Diaz Bou
1627590bc3 core/mvcc: add expect panic in case txn was not found 2025-09-17 12:12:24 +02:00
Pekka Enberg
cb68c953e1 Merge 'core/incremental: Wrap ViewTransactionState in Arc' from Pekka Enberg
Make it Send.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3173
2025-09-17 13:07:54 +03:00
Pekka Enberg
a2f6ff819a Merge 'core/function: Wrap ExtFunc in Arc' from Pekka Enberg
Make it Send.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3172
2025-09-17 13:07:43 +03:00
Pekka Enberg
d3185d2f72 core/storage: Wrap Pager::checkpoint_state in RwLock 2025-09-17 13:04:37 +03:00
Pekka Enberg
bede1229a9 core: Wrap Pager dirty_pages in RwLock
Make it Sync and Send.
2025-09-17 12:55:26 +03:00
Pere Diaz Bou
88307197cc with_header mvcc update rc to arc 2025-09-17 11:50:24 +02:00
Pekka Enberg
182565fe0c core: Wrap MvCursor in Arc<RwLock<>>
Make it Send and Sync.
2025-09-17 12:46:55 +03:00
Pere Diaz Bou
64616dc2ca core/mvcc: introduce with_header for MVCC header update tracking
Currently header changes are tracked through pager by reading page 1.
MVCC has it's own layer to track changes during txn so this commit makes
it so that headers are tracked by each txn separately.

On commit we update the _global_ header which is used to update
`database_size` because pager commits require it to be up to date. This
also makes it _simpler_ to keep track of header updates and update
pager's header accordingly.
2025-09-17 11:42:44 +02:00
Pekka Enberg
e6822d26ab Merge 'core/vtab: Mark VTabModuleImpl as Send and Sync' from Pekka Enberg
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3171
2025-09-17 12:25:12 +03:00
Pekka Enberg
c2b8bb0a2f core/incremental: Wrap ViewTransactionState in Arc
Make it Send.
2025-09-17 12:23:29 +03:00
Pekka Enberg
dcb92f0b50 Merge 'core/vtab: Use AtomicPtr for table_ptr' from Pekka Enberg
Make it Send.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3169
2025-09-17 12:13:18 +03:00
Jussi Saurio
9a2797963a Merge 'Remove LimboResult enum and InsnFunctionStepResult::Busy variant' from Jussi Saurio
We can just use `LimboError::Busy` for both of these.

Reviewed-by: Pekka Enberg <penberg@iki.fi>

Closes #3170
2025-09-17 12:06:54 +03:00
Pekka Enberg
d113c3ac6b core/function: Wrap ExtFunc in Arc
Make it Send.
2025-09-17 11:57:23 +03:00
Pekka Enberg
d7977c25a8 core/vtab: Mark VTabModuleImpl as Send and Sync 2025-09-17 11:48:42 +03:00
Pekka Enberg
6472a71ae7 Merge 'core: Wrap symbol table with RwLock' from Pekka Enberg
Make it Send.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #3168
2025-09-17 11:47:40 +03:00
TcMits
3bd1198bd9 fmt 2025-09-17 15:38:21 +07:00
TcMits
668f1f721c resolve conflict 2025-09-17 15:25:58 +07:00
Jussi Saurio
e066e4f747 clippy 2025-09-17 11:25:50 +03:00
Jussi Saurio
b9ceacc356 Remove InsnFunctionStepResult::Busy
we don't need all these busy variants, let's just handle
LimboError::Busy
2025-09-17 11:22:49 +03:00
Pekka Enberg
e21959d80e core/vtab: Use AtomicPtr for table_ptr
Make it Send.
2025-09-17 11:22:40 +03:00