936ae307b7
We currently have two value types, `Value` and `OwnedValue`. The original thinking was that `Value` is external type and `OwnedValue` is internal type. However, this just results in unnecessary transformation between the types as data crosses the Limbo library boundary. Let's just follow SQLite here and consolidate on a single value type (where `sqlite3_value` is just an alias for the internal `Mem` type). The way this will eventually work is that we can have bunch of pre-allocated `OwnedValue` objects in `ProgramState` and basically return a reference to them all the way to the application itself, which extracts the actual value.
Limbo Simulator
Limbo simulator uses randomized deterministic simulations to test the Limbo database behaviors.
Each simulation begins with a random configurations:
- the database workload distribution(percentages of reads, writes, deletes...),
- database parameters(page size),
- number of reader or writers, etc.
Based on these parameters, we randomly generate interaction plans. Interaction plans consist of statements/queries, and assertions that will be executed in order. The building blocks of interaction plans are:
- Randomly generated SQL queries satisfying the workload distribution,
- Properties, which contain multiple matching queries with assertions indicating the expected result.
An example of a property is the following:
{
"name": "Read your own writes",
"queries": [
"INSERT INTO t1 (id) VALUES (1)",
"SELECT * FROM t1 WHERE id = 1"
],
"assertions": [
"result.rows.length == 1",
"result.rows[0].id == 1"
]
}
The simulator executes the interaction plans in a loop, and checks the assertions. It can add random queries unrelated to the properties without breaking the property invariants to reach more diverse states and respect the configured workload distribution.
The simulator code is broken into 4 main parts:
- Simulator(main.rs): The main entry point of the simulator. It generates random configurations and interaction plans, and executes them.
- Model(model.rs, model/table.rs, model/query.rs): A simpler model of the database, it contains atomic actions for insertion and selection, we use this model while deciding the next actions.
- Generation(generation.rs, generation/table.rs, generation/query.rs, generation/plan.rs): Random generation functions for the database model and interaction plans.
- Properties(properties.rs): Contains the properties that we want to test.
Running the simulator
To run the simulator, you can use the following command:
cargo run
This prompt (in the future) will invoke a clap command line interface to configure the simulator. For now, the simulator runs with the default configurations changing the main.rs file. If you want to see the logs, you can change the RUST_LOG environment variable.
RUST_LOG=info cargo run --bin limbo_sim
Adding new properties
Todo
Adding new generation functions
Todo
Adding new models
Todo
Coverage with Limbo
Todo
Automatic Compatibility Testing with SQLite
Todo
Resources
- (reading) TigerBeetle Deterministic Simulation Testing
- (reading) sled simulation guide (jepsen-proof engineering)
- (video) "Testing Distributed Systems w/ Deterministic Simulation" by Will Wilson
- (video) FF meetup #4 - Deterministic simulation testing
- (code) Hiisi: a proof of concept libSQL written in Rust following TigerBeetle-style with deterministic simulation testing