Our simulator is currently limited to concurrency of one. This
introduces a much less sophisticated DST with focus on finding
concurrency bugs.
Closes#2985
Particularly we were tracing `ImmutableRecord` / `BTreeKey` which would
then trace the bytes of records. These are super super hot paths and I
think we can probably remove even more to under debug assertions so we
dont eat those atomics/branches all the time.
This PR also introduces the `tracing_release` feature, which turns all
`trace!` and `debug!` macro invocations to noops at compile time, and
makes that feature available for all bindings.
it also removes the unused `lru` dependency, and cleans up the makefile
a bit
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#2995
Revives the `MemorySim` PR and fixes a page cache issue where we could
have a unlocked and unloaded page in the page cache after a FaultyQuery.
The page would continue in the cache and could affect other queries as
the `page_cache` is at the `Connection` level.
Depends on #2785Closes#2693
## Changes
- Refactor sql generation to always accept a `Context` trait object so
we can query the current Generation `Opts`. This change allows us to be
more granular in generating our sql statements. It also opens
opportunities for us to add even more knobs to tweak generation as
needed. I tried to make this as generic as possible as I believe this
library can be useful for fuzz testing outside the simulator.
- Introduce `Profile` struct that aggregates the different
configurations needed to execute the simulator. With this Profile struct
we can bias sql generation towards different statements and create
predefined profiles.
`WriteHeavy` Profile:
```rust
Profile {
query: QueryProfile {
gen_opts: Opts {
// TODO: in the future tweak blob size for bigger inserts
// TODO: increase number of rows as well
table: TableOpts {
large_table: LargeTableOpts {
large_table_prob: 0.4,
..Default::default()
},
..Default::default()
},
query: QueryOpts {
insert: InsertOpts {
min_rows: NonZeroU32::new(5).unwrap(),
max_rows: NonZeroU32::new(11).unwrap(),
},
..Default::default()
},
..Default::default()
},
select_weight: 30,
insert_weight: 70,
delete_weight: 0,
update_weight: 0,
..Default::default()
},
..Default::default()
};
```
As you can see we disable the `delete` and `update` weights, decrease
`select` and increase `insert` weights. This means that we disable
updates and deletes in favor of inserting more data and checking the
validity of the database with fewer select statements.
- `Profile` and `Opts` are validated with `garde` and can generate json
schemas with `schemars` so that we can have editor integration when
creating new profiles to play with.
- Added some docs in the README explaining how you can add LSP
integration for the Json config by generating a `JsonSchema` file
Closes#2852
This patch adds support for per page encryption. The code is of alpha
quality, was to test my hypothesis. All the encryption code is gated
behind a `encryption` flag. To play with it, you can do:
```sh
cargo run --features encryption -- database.db
turso> PRAGMA key='turso_test_encryption_key_123456';
turso> CREATE TABLE t(v);
```
Right now, most stuff is hard coded. We use AES GCM 256. This
information is not stored anywhere, but in future versions we will start
saving this info in the file. When writing to disk, we will generate a
cryptographically secure random salt, use that to encrypt the page. Then
we will store the authentication tag and the salt in the page itself. To
accommodate this encryption hardcodes reserved space of 28 bytes.
Once the key is set in the connection, we propagate that information to
pager and the WAL, to encrypt / decrypt when reading from disk.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#2567
Let's add an encryption module, hard coded to use AES 256 GCM.
Other required parameters are also hard coded and will be made
configurable in the future PRs.
The module is behind a `encryption` feature flag.
