It is insane that SQLite even allows this.
They actually don't if "defensive mode" is enabled:
"It is always safe to read the schema_version, but changing the
schema_version can cause problems. For this reason, attempts to change
the value of schema_version are a silent no-op when defensive mode is
enabled for a database connection.
Warning: Misuse of this pragma can result in database corruption."
We also update the compat table, which was not updated to reflect
the read version of this pragma being implemented.
This PR make it simpler to block async operations instead of creating
sync equivalent functions. eg.: `get_page_size` and
`get_page_size_async`. All functions should be async by default, and
making them block should more explicit.
Also makes it easier to grep for top-level uses of blocking code without
needing to "go to definition".
Closes#2173
Closes#2165
The following sequence of events is possible:
- init_chunk_heap() called
- flush() gets called, and all chunks start writing to disk
- chunk A status is WriteComplete, so chunk.read() gets called on chunk
A
- chunk A sets its status to WaitingForRead
- some other chunk B is still in WaitingForWrite status after flush()
- for this reason, init_chunk_heap() returns IOResult::IO
- init_chunk_heap() is called again
- we panic because chunk A is in WaitingForRead status
So - we just allow WaitingForRead status in init_chunk_heap() instead.
This panic was caught thanks to Pedro's IO latency enhancement to the
sim!
Reviewed-by: Iaroslav Zeigerman (@izeigerman)
Closes#2166
Closes#2121
There are two important things to point out:
1. The support is incomplete since we yet don't support savepoints in
core.
2. When a txn drops we should call `_finish()` on it, but since async
drop is [unstable](https://github.com/rust-lang/rust/issues/126482) the
best solution that I came up was just forcing the user to explicitly
call `finish()` before any drops.
Closes#2151
I ended up hitting #1974 today and wanted to fix it. I worked with
Claude to generate a more comprehensive set of queries that could fail
aside from just the insert query described in the issue. He got most of
them right - lots of cases were indeed failing. The ones that were
gibberish, he told me I was absolutely right for pointing out they were
bad.
But alas. With the test cases generated, we can work on fixing it. The
place where the assertion was hit, all we need to do there is return
true (but we assert that this is indeed a string literal, it shouldn't
be anything else at this point).
There are then just a couple of places where we need to make sure we
handle double quotes correctly. We already tested for single quotes in a
couple of places, but never for double quotes.
There is one funny corner case where you can just select "col" from tbl,
and if there is no column "col" on the table, that is treated as a
string literal. We handle that too.
Fixes#1974Closes#2152
I ended up hitting #1974 today and wanted to fix it. I worked with
Claude to generate a more comprehensive set of queries that could fail
aside from just the insert query described in the issue. He got most of
them right - lots of cases were indeed failing. The ones that were
gibberish, he told me I was absolutely right for pointing out they were
bad.
But alas. With the test cases generated, we can work on fixing it. The
place where the assertion was hit, all we need to do there is return
true (but we assert that this is indeed a string literal, it shouldn't
be anything else at this point).
There are then just a couple of places where we need to make sure we
handle double quotes correctly. We already tested for single quotes in a
couple of places, but never for double quotes.
There is one funny corner case where you can just select "col" from tbl,
and if there is no column "col" on the table, that is treated as a
string literal. We handle that too.
Fixes#1974
Currently ignored. The reason we are adding it is so that we have
an output that can fit in a single line. This is so we can use it in
tests, and have a predictable output pattern for both sqlite and turso.
The SQLite command line has facilities to ingest things like csv, and
other formats. But here we are, in 2025, and I asked Claude if Turso's
CLI should, in the same vein, have a native MCP server.
Claude told me: "You're absolutely right!" "That's a great insight!"
"That's a fantastic idea!" and then proceeded to help me with the
boilerplate for this beautiful server.
Rust has a crate, mcp_server, that implements an mcp_server trait.
However, that depends on Tokio, and I think that would bloat our binary
too much.
I have also considered implementing an MCP server that operates on a
directory and allows to list many SQLite files, but figured that would
be a good job for a more advanced and specialized server, not for the
one that comes by default with the CLI. Let's go for simple.
Closes#2148
The following sequence of events is possible:
- init_chunk_heap() called
- chunk A status is WriteComplete, so chunk.read() gets called on chunk A
- some other chunk B is in WaitingForWrite status after flush()
- init_chunk_heap() returns IOResult::IO
- init_chunk_heap() is called again
- we panic because chunk A is in WaitingForRead status
So - we just allow WaitingForRead status in init_chunk_heap() instead.
This panic was caught thanks to Pedro's IO latency enhancement to the sim!
Currently we are using `find_cell` which does a binary search on every
insert, even if the cursor is already in the correct place.
1. Remove `find_cell` and use `seek` when inserting instead of
`move_to`.
2. After removing `find_cell`, we now need to be more mindful of when we
do actually require a seek before insertion. This is signified by the
addition of `InsertFlags::REQUIRE_SEEK`. Previously some inserts worked
by accident because we always happened to be on the correct page --
`find_cell()´ only binary searches a single page.
3. After removing `find_cell` we also need to call `next()` after
`Insn::NewRowid`, because `NewRowid` positions us at the old maximum
rowid, and we need to be positioned after it.
4. After removing `find_cell` we also need to leave the cursor
positioned correctly during seeks where we don't find a match.
5. Allow overwrite of an index interior cell, which Closes#1975.
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#1988
Another fix extracted from running simulations on the #1988 branch.
## What
When interior cell replacement happens as described in #2108, we use the
`cursor.prev()` method to locate the largest key in the left subtree.
There was an error during backwards traversal in the `get_prev_record()`
method where the parent's cell index was set as `i32::MAX` but not
properly set to `cell_count + 1` (indicating that rightmost pointer has
been visited).
This meant that if the child page we took the replacement cell from
underflowed and needed balancing, the parent page would now have
`i32::MAX` as its cell index and crash at the point where we determine
based on the parent page which page is going to undergo balancing.
## Fix
This PR fixes the issue by setting the cell index of the parent properly
when visiting the rightmost child. This way the balance procedure will
correctly detect that the rightmost child page is going to undergo
balancing.
### Trivia
The reason `i32::MAX` is used is that the cell count of the page is not
necessarily known at the time it is pushed to the stack.
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#2163
Another fix extracted from running simulations on the #1988 branch.
When interior cell replacement happens as described in #2108,
we use the `cursor.prev()` method to locate the largest key in the
left subtree.
There was an error during backwards traversal in the `get_prev_record()`
method where the parent's cell index was set as `i32::MAX` but not properly
set to `cell_count + 1` (indicating that rightmost pointer has been visited).
The reason `i32::MAX` is used is that the cell count of the page is not
necessarily known at the time it is pushed to the stack.
This PR fixes the issue by setting the cell index of the parent properly
when visiting the rightmost child.
Fixes#2153.
Not so sure if SQLite doesn't rollback in more cases, we should
definitively check this out.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#2154
pread and pwrite is usually less instructions then seek and read. Also
added possibility for io to retry if AGAIN error happens. And made write
to wait for Event::writable
Reviewed-by: Preston Thorpe (@PThorpe92)
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#2010
Closes#91
Replaces #1771 since the previous PR went stale and had too many
conflicts.
The maximum size of the in-memory buffer is determined by the settings
that configure the page-cache size.
In scope:
- Spill the in-memory buffer to a temporary file on disk if its size
exceeds the threshold
- When iterating over the sorter, the next record is picked using heap
populated from sorted chunk files
- Flushing the in-memory buffer and reading from chunk files occur
asynchronously.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#2128
