## Description
Read transactions from logical log and load mvcc store based on the
contents onto transaction 0, which is the special transaction where we
will load all row versions that can be read by all new transactions.
## Todo
- [x] Testing for multiple transactions
- [ ] Lock multiple loading of same logical-log at the same time
- [x] Add column_count to format because if not it is hard to judge from
record alone.
- [x] Trim buffer read
I was thinking to implement the rest on another pr to not increase load
to review this one.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#3323
Changes ALTER TABLE operations to use only MV store and not go through
pager, because pager is only used during checkpoint.
- Exclusive transaction is required for DDL operations: trying to
execute one inside a `BEGIN CONCURRENT` transaction will return an
error. It's simply too complicated for now to try to make them
concurrently transactional.
- Not doing schema changes via pager means that in MV rollback, the
connection must rollback its private schema separately, since pager
rollback is not invoked.
- To simplify MVCC semantics, if any transaction committed a schema
change after a transaction started, it cannot commit and will abort with
`SchemaUpdated` error
- To mimic regular SQLite transaction behavior, if a transaction tries
to promote to exclusive transaction, it will fail with `Busy` error if
there were any committed transactions after the read transaction
started.
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#3333
Closes#3320Closes#3286
In addition to the problem reported in the ticket, 2 more issues were
identified:
1. Renaming a column for a table with a special character in its name
failed with
```
turso> CREATE TABLE `t t`(a);
turso> ALTER TABLE `t t` RENAME COLUMN a TO `a a`;
thread 'main' panicked at core/vdbe/execute.rs:7870:14:
table being renamed should be in schema
```
2. The renamed table in the `sql` column of `sqlite_schema` was not
reflected correctly after renaming:
```
turso> select * from sqlite_schema;
┌───────┬──────┬──────────┬──────────┬──────────────────────┐
│ type │ name │ tbl_name │ rootpage │ sql │
├───────┼──────┼──────────┼──────────┼──────────────────────┤
│ table │ t t │ t t │ 2 │ CREATE TABLE t t (a) │
└───────┴──────┴──────────┴──────────┴──────────────────────┘
```
3. `sql` for indexes was not reflected correctly after renaming a column
that contains special characters:
```
turso> ALTER TABLE `t t` RENAME COLUMN `a a` TO `b b`;
turso> SELECT sql FROM sqlite_schema;
┌───────────────────────────────────┐
│ sql │
├───────────────────────────────────┤
│ CREATE TABLE `t t` (`b b`) │
├───────────────────────────────────┤
│ CREATE INDEX idx ON `t t` (`a a`) │
├───────────────────────────────────┤
```
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#3322
In e.g. `SELECT x AS y, y AS x FROM t ORDER BY x;`, the `x` in the
`ORDER BY` should reference t.y, which has been aliased as `x` for this
query. The same goes for GROUP BY, JOIN ON etc. but NOT for WHERE.
Previously we had wrong precedence in `bind_and_rewrite_expr`.
Closes#3281Closes#3290
mvcc: add blocking checkpoint
performs a blocking, truncating checkpoint:
- when started, blocks any other transactions from starting
- writes all row versions to pager that havent already been
checkpointed
- flushes them to wal
- checkpoints wal to db
in the case of added/deleted rows in table id 1 (sqlite schema),
also creates/destroys btrees accordingly
Closes#3263
Because `InteractionPlan::init_plan` and `gen_rng` used an rng with the
same seed initially, it could happen that we could create the first
create query in `init_plan`, and then generate a `DoubleCreateFailure`
property that would generate the exact same previous query, due to the
seed being the same. The easy and quick fix is just to use different
seeds.
Closes#3330Closes#3329Closes#3327Closes#3326Closes#3325Closes#3324
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#3331
performs a blocking, truncating checkpoint:
- when started, blocks any other transactions from starting
- writes all row versions to pager that havent already been
checkpointed
- flushes them to wal
- checkpoints wal to db
in the case of added/deleted rows in table id 1 (sqlite schema),
also creates/destroys btrees accordingly
this assigns "synthetic" root page numbers instead of using the pager
or btree cursors to allocate pages. Actual page allocation will only
happen during checkpoint, since regular MVCC commit bypasses the
pager.
In e.g. `SELECT x AS y, y AS x FROM t ORDER BY x;`, the `x` in the
`ORDER BY` should reference t.y, which has been aliased as `x` for this
query. The same goes for GROUP BY, JOIN ON etc. but NOT for WHERE.
Previously we had wrong precedence in `bind_and_rewrite_expr`.
In the hopes of doing a good job at teaching people what Turso can do, I
am adding built-in manual pages. When the CLI starts, it picks a feature
at random, and tells the user that the feature exists:
```
Turso v0.2.0-pre.8
Enter ".help" for usage hints.
Did you know that Turso supports Change Data Capture? Type .manual cdc to learn more.
This software is ALPHA, only use for development, testing, and experimentation.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database
```
There is a lot we can do to make this feature world class:
- we can automatically compile examples during compile time like rust-
doc, to make sure examples used in the manuals always work
- we can implement scrolling and navigation
- we can document a lot more features
But for now, this is a start!
Reviewed-by: Preston Thorpe <preston@turso.tech>
Closes#3298
Read transactions from logical log and load mvcc store based on the
contents onto transaction 0, which is the special transaction where we
will load all row versions that can be read by all new transactions.
- [ ] Testing for multiple transactions
- [ ] Lock multiple loading of same logical-log at the same time
- [ ] Add column_count to format because if not it is hard to judge from
record alone.
This format is based on previous discussions:
1. Log header
```rust
/// Log's Header, this will be the 64 bytes in any logical log file.
/// Log header is 64 bytes at maximum, fields added must not exceed that size. If it doesn't exceed
/// it, any bytes missing will be padded with zeroes.
struct LogHeader {
version: u8,
salt: u64,
encrypted: u8, // 0 is no
}
```
2. Transaction format:
* Transaction id
* Checksum u64
* Byte size of all rows combined
* Rows
* End marker (offset position after appending buffer)
3. Row format:
```rust
/// Serialize a row_version into on disk format.
/// Format of a "row" (maybe we could change the name because row is not general enough for
/// future type of values):
///
/// * table_id (root page) -> u64
/// * row type -> u8
///
/// (by row type)
/// Delete:
/// * Payload length -> u64
/// * Rowid -> varint
///
/// Insert:
/// * Payload length -> u64
/// * Data size -> varint
/// * Rowid -> varint
/// * Data -> [u8] (data size length)
fn serialize(&self, buffer: &mut Vec<u8>, row_version: &RowVersion) {
```
Closes#3245
In the hopes of doing a good job at teaching people what Turso can do,
I am adding built-in manual pages. When the CLI starts, it picks a
feature at random, and tells the user that the feature exists:
```
Turso v0.2.0-pre.8
Enter ".help" for usage hints.
Did you know that Turso supports Change Data Capture? Type .manual cdc to learn more.
This software is ALPHA, only use for development, testing, and experimentation.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database
```
There is a lot we can do to make this feature world class:
- we can automatically compile examples during compile time like
rust-doc, to make sure examples used in the manuals always work
- we can implement scrolling and navigation
- we can document a lot more features
But for now, this is a start!
