Commit Graph

4574 Commits

Author SHA1 Message Date
TcMits
bfff05faba merge main 2025-09-02 18:25:20 +07:00
Pekka Enberg
15d45e3f68 Merge 'Refactor encryption to manage authentication tag internally' from bit-aloo
This PR updates the internal encryption framework to handle
authentication tags explicitly rather than relying on the underlying
cipher libraries to append/verify them automatically.
closes: #2850

Reviewed-by: Avinash Sajjanshetty (@avinassh)

Closes #2858
2025-09-02 09:44:22 +03:00
Pekka Enberg
7189e98455 Merge 'Unify handling of grouped and ungrouped aggregations' from Piotr Rżysko
The initial commits fix issues and plug gaps between ungrouped and
grouped aggregations.
The final commit consolidates the code that emits `AggStep` to prevent
future disparities between the two.

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2867
2025-09-02 09:11:40 +03:00
Pekka Enberg
0868af29df Merge 'core/printf: support for more basic substitution types' from Luiz Gustavo
Some progress working on `printf` support.
(relevant issue https://github.com/tursodatabase/turso/issues/885)
Implementation of the basic substitution types cited in the `TODO`
comment on the beginning of the file (%i, %x, %X, %o, %e, %E, %c). There
are some others in the sqlite spec which I will implement in a future
PR.
I tried to pay attention to the specific behaviors from sqlite as much
as possible while testing this, but if there's something I missed please
tell me.
Also, I see this code needs to be reorganized already, I'm still
thinking on the best approach to do that without affecting the
ergonomics of new implementations, I'm still learning Rust so this is
not obvious for me right now. I'm open to suggestions about it.

Closes #2868
2025-09-02 09:10:03 +03:00
Pekka Enberg
87d3f74e6e Merge 'Evict page from cache if page is unlocked and unloaded' from Pedro Muniz
Because we can abort a read_page completion, this means a page can be in
the cache but be unloaded and unlocked. However, if we do not evict that
page from the page cache, we will return an unloaded page later which
will trigger assertions later on. This is worsened by the fact that page
cache is not per `Statement`, so you can abort a completion in one
Statement, and trigger some error in the next one if we don't evict the
page in these circumstances.
Also, to propagate IO errors we need to return the Error from
IOCompletions on step.

Closes #2785
2025-09-02 09:08:12 +03:00
Pekka Enberg
d959319b42 Merge 'Use u64 for file offsets in I/O and calculate such offsets in u64' from Preston Thorpe
Using `usize` to compute file offsets caps us at ~16GB on 32-bit
systems. For example, with 4 KiB pages we can only address up to 1048576
pages; attempting the next page overflows a 32-bit usize and can wrap
the write offset, corrupting data. Switching our I/O APIs and offset
math to u64 avoids this overflow on 32-bit targets

Closes #2791
2025-09-02 09:06:49 +03:00
Pekka Enberg
cfaba4ab10 Merge 'Implement libSQL's ALTER COLUMN extension' from Levy A.
Implement `ALTER COLUMN` as described here:
https://github.com/tursodatabase/libsql/blob/main/libsql-
sqlite3/doc/libsql_extensions.md#altering-columns
- [x] Add `ALTER COLUMN` to parser
- [x] Implement `Insn::AlterColumn`
- [x] Add tests

Closes #2814
2025-09-02 09:06:03 +03:00
PThorpe92
e9b50b63fb Return sqlite_version() without being initialized 2025-09-01 13:36:41 -04:00
pedrocarlo
bc707fd9be cleanup + comments 2025-09-01 11:10:40 -03:00
pedrocarlo
53cfae1db4 return Error from step if IO failed 2025-09-01 11:10:39 -03:00
pedrocarlo
6f1eed7aca clippy 2025-09-01 11:10:39 -03:00
pedrocarlo
4618df9d1a because we can abort a read_page completion, this means that the page can be in the cache but be unloaded and unlocked. However, if we do not evict that page from the page cache, we will return an unloaded page later 2025-09-01 11:10:39 -03:00
pedrocarlo
be3f944c4f impl Error for CacheError and propagate it into LimboError 2025-09-01 11:10:39 -03:00
bit-aloo
c70fe79eb8 adjust test cfg and cleanup 2025-09-01 16:21:03 +05:30
bit-aloo
27a6dc95c4 simplify Cipher enum to store wrapper types
- Replace boxed `Aes256Gcm` and `Aegis256Cipher` with direct wrapper types:
  - `Cipher::Aes256Gcm(Aes256GcmCipher)`
  - `Cipher::Aegis256(Aegis256Cipher)`
- Add `as_aead()` method to unify access via `AeadCipher` trait.
- Refactor decrypt_raw and encrypt raw.
- Add decrypt_raw_detached and encrypt raw detached.
2025-09-01 16:19:37 +05:30
bit-aloo
7f3c886154 add Aes256GcmCipher implementing AeadCipher
- Create new `Aes256GcmCipher` wrapper around AES-256-GCM.
- Implement `AeadCipher` trait with both combined and detached modes.
2025-09-01 16:18:49 +05:30
bit-aloo
f11e90c94d refactor Aegis256Cipher to implement AeadCipher 2025-09-01 16:18:22 +05:30
bit-aloo
c685c4e735 Add AeadCipher trait abstraction
- Define a common trait `AeadCipher` for encryption/decryption.
- Provide methods for both "combined" and "detached" encryption modes:
  - encrypt / decrypt
  - encrypt_detached / decrypt_detached
2025-09-01 16:16:41 +05:30
bit-aloo
3a9b5cc6fa simplify aes-gcm imports and add tag size constants 2025-09-01 16:15:57 +05:30
TcMits
6e87b08d64 faster type_from_name 2025-09-01 14:38:38 +07:00
Gaurav Sarma
453cbd3201 Decrypt WAL page while reading raw frames 2025-09-01 15:29:01 +08:00
Pekka Enberg
9d06e0bf8e Merge 'Support encryption for non-4k page size' from
Closes #2734.

Reviewed-by: Avinash Sajjanshetty (@avinassh)

Closes #2860
2025-09-01 10:05:52 +03:00
TcMits
ed1fb4cabc remove unnecessary check 2025-09-01 11:51:51 +07:00
Piotr Rzysko
6f1cd17fcf Consolidate methods emitting AggStep 2025-08-31 13:29:10 +02:00
Piotr Rzysko
cdba1f1b87 Generalize GroupByAggArgumentSource
This is primarily a mechanical change: the enum was moved between files,
renamed, and its comments updated so it is no longer strictly tied to
GROUP BY aggregations.

This prepares the enum for reuse with ungrouped aggregations.
2025-08-31 13:23:12 +02:00
Piotr Rzysko
0a85883ee2 Support external aggregate functions in GROUP BY 2025-08-31 12:02:11 +02:00
Piotr Rzysko
7d179bd9fe Fix handling of multiple arguments in aggregate functions
This bug occurred when arguments were read for the GROUP BY sorter — all
arguments were incorrectly resolved to the first column. Added tests
confirm that aggregates now work correctly both with and without the
sorter.
2025-08-31 12:02:11 +02:00
Piotr Rzysko
3ad4016080 Fix handling of zero-argument grouped aggregations
This commit consolidates the creation of the Aggregate struct, which was
previously handled differently in `prepare_one_select_plan` and
`resolve_aggregates`. That discrepancy caused inconsistent handling of
zero-argument aggregates.

The queries added in the new tests would previously trigger a panic.
2025-08-31 12:02:09 +02:00
TcMits
37f33dc45f add eq/contains/starts_with/ends_with_ignore_ascii_case 2025-08-31 16:18:42 +07:00
Piotr Rzysko
978a78b79a Handle COLLATE clause in grouped aggregations
Previously, it was only applied to ungrouped aggregations.
2025-08-31 06:51:26 +02:00
luizgfc
528cab55c1 core/printf: %o substitution type implementation 2025-08-30 21:55:22 -03:00
luizgfc
38d528537a core/printf: %x and %X substitution types implementation 2025-08-30 21:53:48 -03:00
luizgfc
3f7a7d0e39 core/printf: %c substitution type implementation 2025-08-30 21:49:59 -03:00
luizgfc
078b0aca79 core/printf: %e and %E substitution types implementation 2025-08-30 21:41:59 -03:00
luizgfc
82a5a16478 core/printf: %u substitution type implementation 2025-08-30 21:41:59 -03:00
rajajisai
3409de3c47 Include page size when initializing encryption module
Move default page size to test

Store page_size as usize

clippy

fix import and move const inside test module
2025-08-30 16:13:44 -04:00
William Souza
b1114734d3 support cipher and encryption key URI options 2025-08-30 10:29:41 -03:00
Levy A.
293865c2d6 feat+fix: add tests and restrict altering some constraints 2025-08-30 03:43:31 -03:00
Levy A.
ad639b2b23 fix: reintroduce rename
we don't store the parsed column to replace just the name, this will be
refactored later with a more general approach
2025-08-30 03:10:39 -03:00
Levy A.
5b378e3730 feat: add AlterColumn instruction
also refactor `RenameColumn` to reuse the logic from `AlterColumn`
2025-08-30 03:10:39 -03:00
Levy A.
678ca8d33b feat(parser): add ALTER COLUMN 2025-08-30 03:10:39 -03:00
Pekka Enberg
e1b5f2d948 Merge 'Implement UPSERT' from Preston Thorpe
This PR closes #2019
Implements https://sqlite.org/lang_upsert.html

Closes #2853
2025-08-30 08:54:35 +03:00
Pekka Enberg
b22f184a19 Merge 'Fix column case sensitivity on strict table' from
closes: #2822
```
turso> insert into strict_table values (1);
turso>
```

Closes #2823
2025-08-30 08:45:28 +03:00
Pekka Enberg
ca7f1002b4 Merge 'Change views to use DBSP circuits' from Glauber Costa
Instead of using static elements, use a dynamically generated DBSP-
circuit to keep views.
The DBSP circuit is generated from the logical plan, which only supports
enough for us to generate the DBSP circuit at the moment.
The state of the view is still kept inside the IncrementalView, instead
of materialized at the operator level. As a consequence, this still
depends on us always populating the view at startup. Fixing this is the
next step.

Closes #2815
2025-08-30 08:44:06 +03:00
Pekka Enberg
0c16ca9ce9 Merge 'core/wal: cache file size' from Pere Diaz Bou
Closes #2829
2025-08-30 08:41:58 +03:00
Pekka Enberg
e357909cda Merge 'Propagate decryption error from the callback' from Avinash Sajjanshetty
Reviewed-by: Pedro Muniz (@pedrocarlo)

Closes #2843
2025-08-30 08:40:47 +03:00
PThorpe92
8531560899 Combine rewriting expressions in UPSERT into a single walk of the ast 2025-08-29 22:12:46 -04:00
Preston Thorpe
18a9a38c8e Merge ' core/translate: parse_table remove unnecessary clone of table name ' from Pere Diaz Bou
```

Benchmarking Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou...: Collecting 100 samples in estimated 5.008
Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou...
                        time:   [4.0081 µs 4.0223 µs 4.0364 µs]
                        change: [-2.9298% -2.2538% -1.6786%] (p = 0.00 < 0.05)
                        Performance has improved.
                        ```

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2847
2025-08-29 21:45:58 -04:00
themixednuts
eb93e4edc9 remove to_upper_case in favor of eq_ignore_ascii_case 2025-08-29 20:24:43 -05:00
themixednuts
6ffbdb4908 fix: column case sensitivity on strict table 2025-08-29 20:24:43 -05:00