### The problem:
Sqlite displays the column names of the underlying vtab module when
displaying the `.schema`
Previously limbo omitted this, which makes it difficult for the user to
see what/how many columns the module's table has.
This matches sqlite's behavior by fetching the module's schema when the
schema entry is being inserted in translation.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#1168
Closes#1413 . Basically, SQLite emits a check in a transaction to see
if it is attempting to write. If the db is in read only mode, it throws
an error, else the statement is executed. Mirroring how Rusqlite does
it, I modified the `OpenFlags` to use bitflags to better configure how
we open our VFS. This modification, will enable us to run tests against
the same database in parallel.
Closes#1433
closes #1417
Man chasing this down was much much harder than it should have been.
We very frequently call `read_page` then push the return value onto the
page stack, or otherwise use it without it necessarily needing to not be
'in progress' of IO, so it was tricky to figure out where this was
happening and it had me thinking that it was something wrong with the
changes to `io_uring` on my branch.
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#1418
`Statement` and `Rows` both have a private Arc, implementing clone
avoids users needing to Arc<Mutex> it again.
Reviewed-by: Preston Thorpe (@PThorpe92)
Closes#1412
we had an incorrect optimization in `eliminate_orderby_like_groupby()`
where it could remove e.g. the first term of the ORDER BY if it matched
the first GROUP BY term and the result set was naturally ordered by that
term. this is invalid. see e.g.:
```sql
main branch - BAD: removes the `ORDER BY id` term because the results are naturally ordered by id.
However, this results in sorting the entire thing by last name only!
limbo> select id, last_name, count(1) from users GROUP BY 1,2 order by id, last_name desc limit 3;
┌──────┬───────────┬───────────┐
│ id │ last_name │ count (1) │
├──────┼───────────┼───────────┤
│ 6235 │ Zuniga │ 1 │
├──────┼───────────┼───────────┤
│ 8043 │ Zuniga │ 1 │
├──────┼───────────┼───────────┤
│ 944 │ Zimmerman │ 1 │
└──────┴───────────┴───────────┘
after fix - GOOD:
limbo> select id, last_name, count(1) from users GROUP BY 1,2 order by id, last_name desc limit 3;
┌────┬───────────┬───────────┐
│ id │ last_name │ count (1) │
├────┼───────────┼───────────┤
│ 1 │ Foster │ 1 │
├────┼───────────┼───────────┤
│ 2 │ Salazar │ 1 │
├────┼───────────┼───────────┤
│ 3 │ Perry │ 1 │
└────┴───────────┴───────────┘
I also refactored sorters to always use the ast `SortOrder` instead of boolean vectors, and use the `compare_immutable()` utility we use inside btrees too.
Closes#1365
Previously columns that were indexed were updated only in the
BtreeTable, but not on Index table. This commit basically enables
updates on indexes too if they are needed.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#1428
Previously columns that were indexed were updated only in the
BtreeTable, but not on Index table. This commit basically enables
updates on indexes too if they are needed.
This PR adds `PRAGMA schema_version` to get the value of the schema-
version integer at offset 40 in the database header.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#1427
I noticed when updating a table with a primary key, it would sometimes
set primary key column to null. I believe the problem was due to
incorrect condition that was inconsistent with the comment above: "don't
emit null for pkey of virtual tables."
cc: @PThorpe92
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#1422
Since `page_size` in `DatabaseHeader` can be 1 representing 65526 bytes,
it can't be used it directly. Additionally, we should use `u32` instead
of `u16` or `usize` in other contexts.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#1411
This issue was introduced in #819. However, I believe the solution is
suboptimal because `pragma page_count` can never return 1, which is
inconsistent with SQLite.
<img width="442" alt="image" src="https://github.com/user-
attachments/assets/c772eae7-3e9f-4687-a94a-230deb0eb034" />
To align with SQLite's behavior, we should allocate the first page when
the first schema object is created, rather than immediately after
creating database. And it's always preferable to return an accurate page
count.
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#1407
I noticed when updating a table with a primary key, it would sometimes
set primary key column to null. A primary key can be nullified if it
isn't a rowid alias, meaning it isn't a INTEGER PRIMAR KEY.
I haven't found a way to automate these tests.
<p><img width="361" alt="image" src="https://github.com/user-
attachments/assets/a1563776-97e0-4aa5-844a-b9b23c5273e5" /></p>
<p><img width="279" alt="image" src="https://github.com/user-
attachments/assets/df036951-2649-4835-bffa-f25e6f59bb07" /></p>
Closes#1424
Before this change, the history was only saved when the shell was
interrupted (e.g., Ctrl-C pressed twice). With this change, history is
now also saved when the `.exit` or `.quit` commands are used.
I attempted to add shell tests to cover the changes introduced in this
PR, but emulating a terminal/TTY that would work cross-platform seems to
require significant changes to `TestLimboShell` and `LimboShell`. For
example, the `pty` module [currently doesn't support
Windows](https://bugs.python.org/issue41663). I'm open to experimenting,
but I’m unsure if complicating these classes is worthwhile, as saving
history doesn't seem to be critical.
Additionally, it might be worth considering a refactor of the CLI so
that exit and cleanup operations are performed in one place.
Reviewed-by: Preston Thorpe (@PThorpe92)
Closes#1414
DeleteState had a bit too many unnecessary states so I removed them.
Usually we care about having a different state when I/O is triggered
requiring a state to be stored for later.
Furthermore, there was a bug with op_idx_delete where if balance is
triggered, op_idx_delete wouldn't be re-entrant. So a state machine was
added to prevent that from happening.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#1421
DeleteState had a bit too many unnecessary states so I removed them.
Usually we care about having a different state when I/O is triggered
requiring a state to be stored for later.
Furthermore, there was a bug with op_idx_delete where if balance is
triggered, op_idx_delete wouldn't be re-entrant. So a state machine was
added to prevent that from happening.
```bash
jussi@Jussis-MacBook-Pro limbo % git co main && cargo build --bin limbo --release && hyperfine --shell=none --warmup 5 './target/release/limbo TPC-H.db "select l_orderkey, 3 as revenue, o_orderdate, o_shippriority from lineitem, orders, customer where c_mktsegment = '\''FURNITURE'\'' and c_custkey = o_custkey and l_orderkey = o_orderkey and o_orderdate < cast('\''1995-03-29'\'' as datetime) and l_shipdate > cast('\''1995-03-29'\'' as datetime);"'
...
Benchmark 1: ./target/release/limbo TPC-H.db "select l_orderkey, 3 as revenue, o_orderdate, o_shippriority from lineitem, orders, customer where c_mktsegment = 'FURNITURE' and c_custkey = o_custkey and l_orderkey = o_orderkey and o_orderdate < cast('1995-03-29' as datetime) and l_shipdate > cast('1995-03-29' as datetime);"
Time (mean ± σ): 2.104 s ± 0.006 s [User: 1.952 s, System: 0.151 s]
Range (min … max): 2.094 s … 2.115 s 10 runs
jussi@Jussis-MacBook-Pro limbo % git co move-to-micro-opt && cargo build --bin limbo --release && hyperfine --shell=none --warmup 5 './target/release/limbo TPC-H.db "select l_orderkey, 3 as revenue, o_orderdate, o_shippriority from lineitem, orders, customer where c_mktsegment = '\''FURNITURE'\'' and c_custkey = o_custkey and l_orderkey = o_orderkey and o_orderdate < cast('\''1995-03-29'\'' as datetime) and l_shipdate > cast('\''1995-03-29'\'' as datetime);"'
...
Benchmark 1: ./target/release/limbo TPC-H.db "select l_orderkey, 3 as revenue, o_orderdate, o_shippriority from lineitem, orders, customer where c_mktsegment = 'FURNITURE' and c_custkey = o_custkey and l_orderkey = o_orderkey and o_orderdate < cast('1995-03-29' as datetime) and l_shipdate > cast('1995-03-29' as datetime);"
Time (mean ± σ): 1.883 s ± 0.012 s [User: 1.733 s, System: 0.146 s]
Range (min … max): 1.866 s … 1.908 s 10 runs
```
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#1408
