Commit Graph

8263 Commits

Author SHA1 Message Date
bit-aloo
c685c4e735 Add AeadCipher trait abstraction
- Define a common trait `AeadCipher` for encryption/decryption.
- Provide methods for both "combined" and "detached" encryption modes:
  - encrypt / decrypt
  - encrypt_detached / decrypt_detached
2025-09-01 16:16:41 +05:30
bit-aloo
3a9b5cc6fa simplify aes-gcm imports and add tag size constants 2025-09-01 16:15:57 +05:30
Pekka Enberg
e209a17780 Merge 'Support encryption for raw WAL frames' from Gaurav Sarma
Fixes https://github.com/tursodatabase/turso/issues/2704
The PR decrypts the page referred to by the WAL frame while reading raw
frames.
<img width="923" height="189" alt="Screenshot 2025-08-31 at 12 42 53 AM"
src="https://github.com/user-
attachments/assets/5e353cf3-aae7-4260-9378-ee2a2cde3f69" />

Reviewed-by: Avinash Sajjanshetty (@avinassh)

Closes #2762
2025-09-01 13:14:08 +03:00
Gaurav Sarma
453cbd3201 Decrypt WAL page while reading raw frames 2025-09-01 15:29:01 +08:00
Pekka Enberg
d6543ac25c Merge 'Simulator Profiles' from Pedro Muniz
## Changes
- Refactor sql generation to always accept a `Context` trait object so
we can query the current Generation `Opts`. This change allows us to be
more granular in generating our sql statements. It also opens
opportunities for us to add even more knobs to tweak generation as
needed. I tried to make this as generic as possible as I believe this
library can be useful for fuzz testing outside the simulator.
- Introduce `Profile` struct that aggregates the different
configurations needed to execute the simulator. With this Profile struct
we can bias sql generation towards different statements and create
predefined profiles.
`WriteHeavy` Profile:
```rust
Profile {
            query: QueryProfile {
                gen_opts: Opts {
                    // TODO: in the future tweak blob size for bigger inserts
                    // TODO: increase number of rows as well
                    table: TableOpts {
                        large_table: LargeTableOpts {
                            large_table_prob: 0.4,
                            ..Default::default()
                        },
                        ..Default::default()
                    },
                    query: QueryOpts {
                        insert: InsertOpts {
                            min_rows: NonZeroU32::new(5).unwrap(),
                            max_rows: NonZeroU32::new(11).unwrap(),
                        },
                        ..Default::default()
                    },
                    ..Default::default()
                },
                select_weight: 30,
                insert_weight: 70,
                delete_weight: 0,
                update_weight: 0,
                ..Default::default()
            },
            ..Default::default()
        };
```
As you can see we disable the `delete` and `update` weights, decrease
`select` and increase `insert` weights. This means that we disable
updates and deletes in favor of inserting more data and checking the
validity of the database with fewer select statements.
- `Profile` and `Opts` are validated with `garde` and can generate json
schemas with `schemars` so that we can have editor integration when
creating new profiles to play with.
- Added some docs in the README explaining how you can add LSP
integration for the Json config by generating a `JsonSchema` file

Closes #2852
2025-09-01 10:26:33 +03:00
Pekka Enberg
0a01d4841d Merge 'bindings/java: Implement date, time related methods under JDBC4PreparedStatement' from Kim Seon Woo
- Implement data and time related methods under JDBC4PreparedStatement

Closes #2864
2025-09-01 10:06:16 +03:00
Pekka Enberg
9d06e0bf8e Merge 'Support encryption for non-4k page size' from
Closes #2734.

Reviewed-by: Avinash Sajjanshetty (@avinassh)

Closes #2860
2025-09-01 10:05:52 +03:00
Pekka Enberg
8e5e752722 Merge 'Support cipher and encryption key URI options' from William Souza
Closes #2851

Reviewed-by: Avinash Sajjanshetty (@avinassh)

Closes #2857
2025-09-01 08:25:29 +03:00
김선우
424275b60b Apply lint 2025-08-31 16:15:37 +09:00
rajajisai
975a83c719 Include test 2025-08-30 16:13:44 -04:00
rajajisai
3409de3c47 Include page size when initializing encryption module
Move default page size to test

Store page_size as usize

clippy

fix import and move const inside test module
2025-08-30 16:13:44 -04:00
William Souza
0a6c3872a7 add test for encryption URI options 2025-08-30 15:56:43 -03:00
pedrocarlo
2cc0bb12d7 add Simulator profiles to CI 2025-08-30 13:09:27 -03:00
pedrocarlo
4f2bc96dbe add Faultless profile 2025-08-30 13:07:19 -03:00
Pekka Enberg
3c9dbfb09e Turso 0.1.5-pre.1 2025-08-30 18:28:38 +03:00
pedrocarlo
5881ee71d6 clippy 2025-08-30 12:21:37 -03:00
pedrocarlo
961c0cd282 script to save JsonSchema for editor integration 2025-08-30 12:17:50 -03:00
pedrocarlo
9aac45c3de small docs for profile 2025-08-30 11:31:52 -03:00
pedrocarlo
b9cc556a55 adjust write heavy profile to insert more rows 2025-08-30 11:31:52 -03:00
pedrocarlo
61fa7546c1 fold some SimulatorOpts fields to Profile 2025-08-30 11:31:52 -03:00
pedrocarlo
463eb1fefd simplify profile weights for writes 2025-08-30 11:31:52 -03:00
pedrocarlo
2f237fdcfd adjust remaining calculation to use the profile 2025-08-30 11:31:52 -03:00
pedrocarlo
962666831b read Profile file from path or use predefined profiles 2025-08-30 11:31:52 -03:00
pedrocarlo
a1407869d4 add serde, schemars and garde to profiles and options 2025-08-30 11:31:52 -03:00
pedrocarlo
faa943fc76 reduce cloning for pick_unique 2025-08-30 11:31:52 -03:00
pedrocarlo
bc6976fd33 add Select and Insert generation options 2025-08-30 11:31:52 -03:00
pedrocarlo
06b923d0c1 adjust simulator to use correct trait signature 2025-08-30 11:31:52 -03:00
pedrocarlo
9bc8bdb279 all Arbitrary traits need to pass a GenerationContext 2025-08-30 11:31:52 -03:00
pedrocarlo
1a8b78afd8 create ArbitraryContext and ArbitraryContextFrom traits to pass generation context + start implementing them in Table + FromClause 2025-08-30 11:31:52 -03:00
pedrocarlo
e0552629e3 create Generation Options structs 2025-08-30 11:31:52 -03:00
pedrocarlo
ef16bc4cfb add profiles together 2025-08-30 11:31:52 -03:00
pedrocarlo
918c2a3f69 extend latency profile + impl Default manually 2025-08-30 11:31:52 -03:00
pedrocarlo
19d9003cd7 create profiles folder 2025-08-30 11:31:52 -03:00
William Souza
b1114734d3 support cipher and encryption key URI options 2025-08-30 10:29:41 -03:00
Pekka Enberg
e1b5f2d948 Merge 'Implement UPSERT' from Preston Thorpe
This PR closes #2019
Implements https://sqlite.org/lang_upsert.html

Closes #2853
2025-08-30 08:54:35 +03:00
Pekka Enberg
13057c8013 testing: Improve insert.test for STRICT mode type case insensitivity 2025-08-30 08:52:05 +03:00
Pekka Enberg
b22f184a19 Merge 'Fix column case sensitivity on strict table' from
closes: #2822
```
turso> insert into strict_table values (1);
turso>
```

Closes #2823
2025-08-30 08:45:28 +03:00
Pekka Enberg
ca7f1002b4 Merge 'Change views to use DBSP circuits' from Glauber Costa
Instead of using static elements, use a dynamically generated DBSP-
circuit to keep views.
The DBSP circuit is generated from the logical plan, which only supports
enough for us to generate the DBSP circuit at the moment.
The state of the view is still kept inside the IncrementalView, instead
of materialized at the operator level. As a consequence, this still
depends on us always populating the view at startup. Fixing this is the
next step.

Closes #2815
2025-08-30 08:44:06 +03:00
Pekka Enberg
e684121469 Merge 'CLI: implement Line output .mode' from Andrey Oskin
This is implementation of #2801, basically repeating of `sqlite` output.
1. I do not want any bike-shedding, so I am totally fine with sqlite
version. For me it was mainly fun trying to work with Rust. On the other
hand, if it will be needed to do it somehow differently, than I am
totally fine with that. Even more fun :-)
2. I do not know where tests for this kind of things are, sorry. If
tests are needed and somebody can point to me to the place where they
can be found, I'll be more than happy to add them as well.
Currently, I can only show how it looks like this way:
```bash
> target/debug/tursodb -m records ../db1
Turso v0.1.4
Enter ".help" for usage hints.
This software is ALPHA, only use for development, testing, and experimentation.
turso> CREATE TABLE test(x INT, some_text TEXT, val DOUBLE);
turso> INSERT INTO test VALUES (100, "very important info", 23.12), (2, NULL, 3.14159265), (98, "yet another very important and long text record", NULL);
turso> SELECT * FROM test;
        x = 100
some_text = very important info
      val = 23.12

        x = 2
some_text = 
      val = 3.14159265

        x = 98
some_text = yet another very important and long text record
      val = 
```

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2835
2025-08-30 08:43:31 +03:00
Pekka Enberg
0c16ca9ce9 Merge 'core/wal: cache file size' from Pere Diaz Bou
Closes #2829
2025-08-30 08:41:58 +03:00
Pekka Enberg
e357909cda Merge 'Propagate decryption error from the callback' from Avinash Sajjanshetty
Reviewed-by: Pedro Muniz (@pedrocarlo)

Closes #2843
2025-08-30 08:40:47 +03:00
Pekka Enberg
352099e8ae Merge 'add sqlite integrity check back' from Pedro Muniz
Closes #2719

Reviewed-by: Avinash Sajjanshetty (@avinassh)

Closes #2849
2025-08-30 08:40:21 +03:00
Arkoniak
2c0e3cf593 feat: consistent interrupt event processing (#2801) 2025-08-30 06:57:14 +03:00
Arkoniak
cb602d960d feat: records output (#2801) 2025-08-30 06:57:14 +03:00
PThorpe92
8531560899 Combine rewriting expressions in UPSERT into a single walk of the ast 2025-08-29 22:12:46 -04:00
Preston Thorpe
c844d91866 Merge 'Remove some code duplication in the CLI' from Preston Thorpe
Just some minor refactoring

Closes #2845
2025-08-29 21:48:53 -04:00
Preston Thorpe
18a9a38c8e Merge ' core/translate: parse_table remove unnecessary clone of table name ' from Pere Diaz Bou
```

Benchmarking Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou...: Collecting 100 samples in estimated 5.008
Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou...
                        time:   [4.0081 µs 4.0223 µs 4.0364 µs]
                        change: [-2.9298% -2.2538% -1.6786%] (p = 0.00 < 0.05)
                        Performance has improved.
                        ```

Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2847
2025-08-29 21:45:58 -04:00
themixednuts
eb93e4edc9 remove to_upper_case in favor of eq_ignore_ascii_case 2025-08-29 20:24:43 -05:00
themixednuts
f9b0c0aa27 chore: add update test to lowercase
Instead of making a new test that would be the same, just updated this one to show that sqlite and turso (with this pr) match and isnt case sensitive in strict tables.
2025-08-29 20:24:43 -05:00
themixednuts
6ffbdb4908 fix: column case sensitivity on strict table 2025-08-29 20:24:43 -05:00