Table ID is an opaque identifier that is only meaningful to the MV store.
Each checkpointed MVCC table corresponds to a single B-tree on the pager,
which naturally has a root page.
We cannot use root page as the MVCC table ID directly because:
- We assign table IDs during MVCC commit, but
- we commit pages to the pager only during checkpoint
which means the root page is not easily knowable ahead of time.
Hence, we:
- store the mapping between table id and btree rootpage
- sqlite_schema rows will have a negative rootpage column if the
table has not been checkpointed yet.
closes#3285
maybe adding seperate func for it is stupid but i kept running into
issues with not closing some random `}` and it got annoying real quick
so i just moved tht into its own func. - and as i am using same logic in
3 places i think it's ok.
Closes#3412
MVCC is like the annoying younger cousin (I know because I was him) that
needs to be treated differently. MVCC requires us to use root_pages that
might not be allocated yet, and the plan is to use negative root_pages
for that case. Therefore, we need i64 in order to fit this change.
This PR moves part of string normalization to the parser layer.
Now, we dequote and unescape values in the parser, but we still need to
lowercase them for proper ignore-case comparison logic in the planner.
The reason to not lowercase in the parser is following:
1. SQLite (and tursodb) have ident->string conversion rule and by
lowercasing value early we will loose original representation
2. Some things like column names are preserve the case right now and we
better to not change this behaviour.
Closes#3344
fixes#1976
and #1605
```zsh
turso> DROP TABLE IF EXISTS t;
CREATE TABLE t (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT
);
turso> INSERT INTO t (name) VALUES ('A'); SELECT * FROM sqlite_sequence;
┌──────┬─────┐
│ name │ seq │
├──────┼─────┤
│ t │ 1 │
└──────┴─────┘
turso> DROP TABLE IF EXISTS t;
CREATE TABLE t (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT
);
turso> INSERT INTO t (name) VALUES ('A'); SELECT * FROM sqlite_sequence;
┌──────┬─────┐
│ name │ seq │
├──────┼─────┤
│ t │ 1 │
└──────┴─────┘
turso> INSERT INTO t (name) VALUES ('A'); SELECT * FROM sqlite_sequence;
┌──────┬─────┐
│ name │ seq │
├──────┼─────┤
│ t │ 2 │
└──────┴─────┘
turso>
```
Reviewed-by: Preston Thorpe <preston@turso.tech>
Closes#2983
Changes ALTER TABLE operations to use only MV store and not go through
pager, because pager is only used during checkpoint.
- Exclusive transaction is required for DDL operations: trying to
execute one inside a `BEGIN CONCURRENT` transaction will return an
error. It's simply too complicated for now to try to make them
concurrently transactional.
- Not doing schema changes via pager means that in MV rollback, the
connection must rollback its private schema separately, since pager
rollback is not invoked.
- To simplify MVCC semantics, if any transaction committed a schema
change after a transaction started, it cannot commit and will abort with
`SchemaUpdated` error
- To mimic regular SQLite transaction behavior, if a transaction tries
to promote to exclusive transaction, it will fail with `Busy` error if
there were any committed transactions after the read transaction
started.
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#3333
Closes#3320Closes#3286
In addition to the problem reported in the ticket, 2 more issues were
identified:
1. Renaming a column for a table with a special character in its name
failed with
```
turso> CREATE TABLE `t t`(a);
turso> ALTER TABLE `t t` RENAME COLUMN a TO `a a`;
thread 'main' panicked at core/vdbe/execute.rs:7870:14:
table being renamed should be in schema
```
2. The renamed table in the `sql` column of `sqlite_schema` was not
reflected correctly after renaming:
```
turso> select * from sqlite_schema;
┌───────┬──────┬──────────┬──────────┬──────────────────────┐
│ type │ name │ tbl_name │ rootpage │ sql │
├───────┼──────┼──────────┼──────────┼──────────────────────┤
│ table │ t t │ t t │ 2 │ CREATE TABLE t t (a) │
└───────┴──────┴──────────┴──────────┴──────────────────────┘
```
3. `sql` for indexes was not reflected correctly after renaming a column
that contains special characters:
```
turso> ALTER TABLE `t t` RENAME COLUMN `a a` TO `b b`;
turso> SELECT sql FROM sqlite_schema;
┌───────────────────────────────────┐
│ sql │
├───────────────────────────────────┤
│ CREATE TABLE `t t` (`b b`) │
├───────────────────────────────────┤
│ CREATE INDEX idx ON `t t` (`a a`) │
├───────────────────────────────────┤
```
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#3322
performs a blocking, truncating checkpoint:
- when started, blocks any other transactions from starting
- writes all row versions to pager that havent already been
checkpointed
- flushes them to wal
- checkpoints wal to db
in the case of added/deleted rows in table id 1 (sqlite schema),
also creates/destroys btrees accordingly
this assigns "synthetic" root page numbers instead of using the pager
or btree cursors to allocate pages. Actual page allocation will only
happen during checkpoint, since regular MVCC commit bypasses the
pager.
