Changes ALTER TABLE operations to use only MV store and not go through
pager, because pager is only used during checkpoint.
- Exclusive transaction is required for DDL operations: trying to
execute one inside a `BEGIN CONCURRENT` transaction will return an
error. It's simply too complicated for now to try to make them
concurrently transactional.
- Not doing schema changes via pager means that in MV rollback, the
connection must rollback its private schema separately, since pager
rollback is not invoked.
- To simplify MVCC semantics, if any transaction committed a schema
change after a transaction started, it cannot commit and will abort with
`SchemaUpdated` error
- To mimic regular SQLite transaction behavior, if a transaction tries
to promote to exclusive transaction, it will fail with `Busy` error if
there were any committed transactions after the read transaction
started.
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#3333
Closes#3320Closes#3286
In addition to the problem reported in the ticket, 2 more issues were
identified:
1. Renaming a column for a table with a special character in its name
failed with
```
turso> CREATE TABLE `t t`(a);
turso> ALTER TABLE `t t` RENAME COLUMN a TO `a a`;
thread 'main' panicked at core/vdbe/execute.rs:7870:14:
table being renamed should be in schema
```
2. The renamed table in the `sql` column of `sqlite_schema` was not
reflected correctly after renaming:
```
turso> select * from sqlite_schema;
┌───────┬──────┬──────────┬──────────┬──────────────────────┐
│ type │ name │ tbl_name │ rootpage │ sql │
├───────┼──────┼──────────┼──────────┼──────────────────────┤
│ table │ t t │ t t │ 2 │ CREATE TABLE t t (a) │
└───────┴──────┴──────────┴──────────┴──────────────────────┘
```
3. `sql` for indexes was not reflected correctly after renaming a column
that contains special characters:
```
turso> ALTER TABLE `t t` RENAME COLUMN `a a` TO `b b`;
turso> SELECT sql FROM sqlite_schema;
┌───────────────────────────────────┐
│ sql │
├───────────────────────────────────┤
│ CREATE TABLE `t t` (`b b`) │
├───────────────────────────────────┤
│ CREATE INDEX idx ON `t t` (`a a`) │
├───────────────────────────────────┤
```
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#3322
In e.g. `SELECT x AS y, y AS x FROM t ORDER BY x;`, the `x` in the
`ORDER BY` should reference t.y, which has been aliased as `x` for this
query. The same goes for GROUP BY, JOIN ON etc. but NOT for WHERE.
Previously we had wrong precedence in `bind_and_rewrite_expr`.
Closes#3281Closes#3290
performs a blocking, truncating checkpoint:
- when started, blocks any other transactions from starting
- writes all row versions to pager that havent already been
checkpointed
- flushes them to wal
- checkpoints wal to db
in the case of added/deleted rows in table id 1 (sqlite schema),
also creates/destroys btrees accordingly
this assigns "synthetic" root page numbers instead of using the pager
or btree cursors to allocate pages. Actual page allocation will only
happen during checkpoint, since regular MVCC commit bypasses the
pager.
In e.g. `SELECT x AS y, y AS x FROM t ORDER BY x;`, the `x` in the
`ORDER BY` should reference t.y, which has been aliased as `x` for this
query. The same goes for GROUP BY, JOIN ON etc. but NOT for WHERE.
Previously we had wrong precedence in `bind_and_rewrite_expr`.
This format is based on previous discussions:
1. Log header
```rust
/// Log's Header, this will be the 64 bytes in any logical log file.
/// Log header is 64 bytes at maximum, fields added must not exceed that size. If it doesn't exceed
/// it, any bytes missing will be padded with zeroes.
struct LogHeader {
version: u8,
salt: u64,
encrypted: u8, // 0 is no
}
```
2. Transaction format:
* Transaction id
* Checksum u64
* Byte size of all rows combined
* Rows
* End marker (offset position after appending buffer)
3. Row format:
```rust
/// Serialize a row_version into on disk format.
/// Format of a "row" (maybe we could change the name because row is not general enough for
/// future type of values):
///
/// * table_id (root page) -> u64
/// * row type -> u8
///
/// (by row type)
/// Delete:
/// * Payload length -> u64
/// * Rowid -> varint
///
/// Insert:
/// * Payload length -> u64
/// * Data size -> varint
/// * Rowid -> varint
/// * Data -> [u8] (data size length)
fn serialize(&self, buffer: &mut Vec<u8>, row_version: &RowVersion) {
```
Closes#3245
fixes#3231
```zsh
❯ sqlite3
SQLite version 3.50.4 2025-07-30 19:33:53
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite> CREATE TABLE t1 (a);
ALTER TABLE t1 ADD COLUMN a;
Parse error: duplicate column name: a
sqlite> ALTER TABLE t1 ADD COLUMN name varchar(255);
SELECT sql FROM sqlite_schema WHERE name = 't1';
CREATE TABLE t1 (a, name varchar(255))
sqlite>
```
```zsh
turso>
turso> CREATE TABLE t1 (a);
ALTER TABLE t1 ADD COLUMN a;
x Parse error: duplicate column name: a
turso> ALTER TABLE t1 ADD COLUMN name varchar(255);
SELECT sql FROM sqlite_schema WHERE name = 't1';
┌─────────────────────────────────────────┐
│ sql │
├─────────────────────────────────────────┤
│ CREATE TABLE t1 (a, name varchar (255)) │
└─────────────────────────────────────────┘
turso>
```
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#3249
This PR implements the `Sequence` and `SequenceTest` opcodes, although
does not yet add plumbing to emit the latter.
SQLite has two distinct mechanisms that determine the final row order
with aggregates:
Traversal order of GROUP BY, and ORDER BY tiebreaking. When ORDER BY
contains only aggregate expressions and/or constants, SQLite has no
extra tiebreak key, but when ORDER BY mixes aggregate and non-aggregate
terms, SQLite adds an implicit, stable row `sequence` so “ties” respect
the input order.
This PR also fixes an issue with a query like the following:
```sql
SELECT u.first_name, COUNT(*) AS c
FROM users u
JOIN orders o ON o.user_id = u.id
GROUP BY u.first_name
ORDER BY c DESC;
```
Because ORDER BY has only an aggregate (COUNT(*) DESC) and no non-
aggregate terms, SQLite traverses the group key (u.first_name) in DESC
order in this case, so ties on c naturally appear with group keys in
descending order.
Previously tursodb would return the group key sorted in ASC order,
because it was used in all cases as the default
Closes#3287
