This PR fixes two bugs in `CREATE VIRTUAL TABLE` (see individual commits
for details).
I added a test in `extensions.py` instead of the TCL test suite because
it's currently more convenient - there’s no existing framework for
loading extensions in TCL tests. However, I believe the test should
eventually be moved to TCL, as it verifies behavior expected to be
compatible with SQLite and is independent of any specific extension.
I've seen a PR proposing to migrate TCL tests to Rust, so please let me
know if moving this test to TCL would still be valuable.
Reviewed-by: Preston Thorpe (@PThorpe92)
Closes#1471
After reading #1123, I wanted to see what optimizations I could do.
Sqlite optimizes `count` aggregation for the following case: `SELECT
count() FROM <tbl>`. This is so widely used, that they made an
optimization just for it in the form of the `COUNT` opcode.
This PR thus implements this optimization by creating the `COUNT`
opcode, and checking in the select emitter if we the query is a Simple
Count Query. If it is, we just emit the Opcode instead of going through
a Rewind loop, saving on execution time.
The screenshots below show a huge decrease in execution time.
- **Main**
<img width="383" alt="image" src="https://github.com/user-
attachments/assets/99a9dec4-e7c5-41db-ba67-4eafa80dd2e6" />
- **Count Optimization**
<img width="435" alt="image" src="https://github.com/user-
attachments/assets/e93b3233-92e6-4736-aa60-b52b2477179f" />
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#1443
Fixes a runtime panic caused by failing to halt the virtual machine
after executing CREATE VIRTUAL TABLE IF NOT EXISTS.
Previously resulted in:
thread 'main' panicked at core/vdbe/mod.rs:408:52:
index out of bounds: the len is 3 but the index is 3
This commit adds suport for DROP INDEX.
Bytecode produced by this commit differs from SQLITE's bytecode, main
reason we don't do autovacuum or repacking of pages like SQLITE does.
Closes#1280Closes#1444
The following code reproduces the leak (memory usage increases over
time):
```rust
#[tokio::main]
async fn main() {
let db = Builder::new_local(":memory:").build().await.unwrap();
let conn = db.connect().unwrap();
conn.execute("SELECT load_extension('./target/debug/liblimbo_series');", ())
.await
.unwrap();
loop {
conn.execute("SELECT * FROM generate_series(1,10,2);", ())
.await
.unwrap();
}
}
```
After switching to the system allocator, the leak becomes detectable
with Valgrind:
```
32,000 bytes in 1,000 blocks are definitely lost in loss record 24 of 24
at 0x538580F: malloc (vg_replace_malloc.c:446)
by 0x62E15FA: alloc::alloc::alloc (alloc.rs:99)
by 0x62E172C: alloc::alloc::Global::alloc_impl (alloc.rs:192)
by 0x62E1530: allocate (alloc.rs:254)
by 0x62E1530: alloc::alloc::exchange_malloc (alloc.rs:349)
by 0x62E0271: new<limbo_series::GenerateSeriesCursor> (boxed.rs:257)
by 0x62E0271: open_GenerateSeriesVTab (lib.rs:19)
by 0x425D8FA: limbo_core::VirtualTable::open (lib.rs:732)
by 0x4285DDA: limbo_core::vdbe::execute::op_vopen (execute.rs:890)
by 0x42351E8: limbo_core::vdbe::Program::step (mod.rs:396)
by 0x425C638: limbo_core::Statement::step (lib.rs:610)
by 0x40DB238: limbo::Statement::execute::{{closure}} (lib.rs:181)
by 0x40D9EAF: limbo::Connection::execute::{{closure}} (lib.rs:109)
by 0x40D54A1: example::main::{{closure}} (example.rs:26)
```
Interestingly, when using mimalloc, neither Valgrind nor mimalloc’s
internal statistics report the leak.
Reviewed-by: Preston Thorpe (@PThorpe92)
Closes#1447
This PR is an enabler for our (Coming Soon ™️ ) join reordering
optimizer -- simply adds the notion of a join order to the current query
execution. This PR does not do any join ordering -- the join order is
always the same as expressed in the SQL query.
Reviewed-by: Preston Thorpe (@PThorpe92)
Closes#1439
The following code reproduces the leak, with memory usage increasing
over time:
```
#[tokio::main]
async fn main() {
let db = Builder::new_local(":memory:").build().await.unwrap();
let conn = db.connect().unwrap();
conn.execute("SELECT load_extension('./target/debug/liblimbo_series');", ())
.await
.unwrap();
loop {
conn.execute("SELECT * FROM generate_series(1,10,2);", ())
.await
.unwrap();
}
}
```
After switching to the system allocator, the leak becomes detectable
with Valgrind:
```
32,000 bytes in 1,000 blocks are definitely lost in loss record 24 of 24
at 0x538580F: malloc (vg_replace_malloc.c:446)
by 0x62E15FA: alloc::alloc::alloc (alloc.rs:99)
by 0x62E172C: alloc::alloc::Global::alloc_impl (alloc.rs:192)
by 0x62E1530: allocate (alloc.rs:254)
by 0x62E1530: alloc::alloc::exchange_malloc (alloc.rs:349)
by 0x62E0271: new<limbo_series::GenerateSeriesCursor> (boxed.rs:257)
by 0x62E0271: open_GenerateSeriesVTab (lib.rs:19)
by 0x425D8FA: limbo_core::VirtualTable::open (lib.rs:732)
by 0x4285DDA: limbo_core::vdbe::execute::op_vopen (execute.rs:890)
by 0x42351E8: limbo_core::vdbe::Program::step (mod.rs:396)
by 0x425C638: limbo_core::Statement::step (lib.rs:610)
by 0x40DB238: limbo::Statement::execute::{{closure}} (lib.rs:181)
by 0x40D9EAF: limbo::Connection::execute::{{closure}} (lib.rs:109)
by 0x40D54A1: example::main::{{closure}} (example.rs:26)
```
Interestingly, when using mimalloc, neither Valgrind nor mimalloc’s
internal statistics report the leak.
This commit adds suport for DROP INDEX.
Bytecode produced by this commit differs from SQLITE's bytecode, main
reason we don't do autovacuum or repacking of pages like SQLITE does.
### The problem:
Sqlite displays the column names of the underlying vtab module when
displaying the `.schema`
Previously limbo omitted this, which makes it difficult for the user to
see what/how many columns the module's table has.
This matches sqlite's behavior by fetching the module's schema when the
schema entry is being inserted in translation.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#1168
Closes#1413 . Basically, SQLite emits a check in a transaction to see
if it is attempting to write. If the db is in read only mode, it throws
an error, else the statement is executed. Mirroring how Rusqlite does
it, I modified the `OpenFlags` to use bitflags to better configure how
we open our VFS. This modification, will enable us to run tests against
the same database in parallel.
Closes#1433
