We use relaxed ordering in a lot of places where we really need to
ensure all CPUs see the write. Switch to sequential consistency, unless
acquire/release is explicitly used. If there are places that can be
optimized, we can switch to relaxed case-by-case, but have a comment
explaning *why* it is safe.
Closes#3193
We use relaxed ordering in a lot of places where we really need to
ensure all CPUs see the write. Switch to sequential consistency, unless
acquire/release is explicitly used. If there are places that can be
optimized, we can switch to relaxed case-by-case, but have a comment
explaning *why* it is safe.
Commit eeab6d5ce ("stress: Retry sync on error to avoid a panic") only
enabled sync retry in the schema creation path. Let's add it to the
stress test loop too.
Spotted during Antithesis runs.
Closes#3192
Hi 👋 and thanks for the great project!
Related to my comment on #85
Here I implemented 3 missing APIs and added a GH Action job to verify
the results.
Let me know if I missed anything and if this is aligned with the
direction of the project.
Reviewed-by: Nikita Sivukhin (@sivukhin)
Closes#2982
Commit eeab6d5ce ("stress: Retry sync on error to avoid a panic") only
enabled sync retry in the schema creation path. Let's add it to the
stress test loop too.
Spotted during Antithesis runs.
This PR eliminates blocking IO calls (`io.block` /
`io.wait_for_completion`) from common execution paths
I need this because I am trying to make turso in browser work smoothly
and current paradigm doesn't work well as it use same connection from
different threads. But, in order to run DB on main thread only (expect
IO) - we need to eliminate all blocking calls (they block main thread
and it can't exit from this state).
This PR eliminates blocking behaviour from following places:
1. `append_frames` now fully async but `prepare_wal_start` /
`prepare_wal_finish` must be called before it in order to ensure that
WAL header is initialized
2. `op_transaction` is non-blocking and read db header async
3. `op_sorter_open` is non-blocking and read db header async in the
beginnig of execution
4. `op_open_ephemeral` is non-blocking and read db header async in the
beginning of execution (note, that I am also removed weird logic which
read page size from the empty ephemeral DB file)
5. `op_checkpoint` is non blocking and checkpoint itself now have more
complex state machine to handle previously blocking behaviour
Closes#3179
If encryption is arg is passed, then we run whopper tests with
encryptions. We randomly generate cipher and key, and run whopper. They
are also printed for debugging and analysis.
Also, updated the corresponding scripts. So now you can do:
```bash
./whopper/bin/run --enable-encryption
or
./whopper/bin/explore --enable-encryption
```
Closes#3183
Ongoing tests for [turso-go](https://github.com/tursodatabase/turso-go)
have unearthed a couple more issues
closes#3187
### Number 1:
We were getting something like:
```sql
sqlite_autoindex_`databases`_2
```
when creating autoindex for table in Gorm (gorm is notorious for
backticks everywhere), because of not normalizing the column name when
creating autoindex.
### Number 2:
When creating table with `PRIMARY KEY AUTOINCREMENT`, we were still
creating the index, but it wasn't properly handled in
`populate_indices`, because we are doing the following:
```rust
if column.primary_key && unique_set.is_primary_key {
if column.is_rowid_alias {
// rowid alias, no index needed
continue; // continues, but doesn't consume it..
}
```
So if we created such an index entry for the AUTOINCREMENT... we would
trip this:
```rust
assert!(automatic_indexes.is_empty(), "all automatic indexes parsed from sqlite_schema should have been consumed, but {} remain", automatic_indexes.len());
```
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#3186
this PR improves 3-6% for `prepare` benchmark without slowing down
others. After this PR we don't have to store `InsnFunction` in
`Program` and `ProgramBuilder` anymore, because `to_function` will
return result without matching.
Reviewed-by: Preston Thorpe <preston@turso.tech>
Closes#3098
This is the start of the work to add transactional properties to the
Simulator. Before this PR, we only executed the queries on one
connection, now the default is to run with 10 connections. Each
connection is selected at random to run the database queries.
A lot of the code was hardcoded to work with 1 connection only, so most
of the refactor came from me adjusting the `Interaction` and
`Interactions` structs to hold a connection index. As I had to refactor
a lot of code, I took the opportunity to deduplicate code that across
`run_simulation_default`, `watch::run_simulation`,
`differential::run_simulation`, `doublecheck::run_simulation`.
In my next PR, I intend to add transaction isolation checking which will
involve some more refactoring to ensure the simulator correctly tracks
transaction state and table visibility.
And in a future PR, we can extend this further with concurrent
transactions as well
Closes#3163
Currently header changes are tracked through pager by reading page 1.
MVCC has it's own layer to track changes during txn so this commit makes
it so that headers are tracked by each txn separately.
On commit we update the _global_ header which is used to update
`database_size` because pager commits require it to be up to date. This
also makes it _simpler_ to keep track of header updates and update
pager's header accordingly.
This PR is needed in order to make logical log work because we don't
want to rely on pager as much as possible!
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#3156
This PR fixes bugs found in the [turso-
go](https://github.com/tursodatabase/turso-go) driver with UPSERT clause
earlier, where `Gorm` will (obviously) use Expr::Variable's as well as
use quotes for `Expr::Qualified` in the tail end of an UPSERT statement.
Example:
```sql
INSERT INTO users (a,b,c) VALUES (?,?,?) ON CONFLICT (`users`.`a`) DO UPDATE SET b = `excluded`.`b`, a = ?;
```
and previously we were not properly calling `rewrite_expr`, which was
not properly setting the anonymous `Expr::Variable` to `__param_N` named
parameter, so it would ignore it completely, then return the wrong # of
parameters.
Also, we didn't handle quoted "`excluded`.`x`", so it would panic in the
optimizer that Qualified should have been rewritten earlier.
Closes#3157
