This PR fixes#666
Limbo was treating all single byte integers properly in `core/types.rs`,
but when converted in `sqlite3_ondisk`, it was losing the sign for 8 bit
integers, treating them all as unsigned. Sqlite3 specifies in their file
format that the `1` record serial type is "Value is an 8-bit twos-
complement integer." https://www.sqlite.org/fileformat.html
We now properly match sqlite3's output
Closes#667
Give the user the option of choosing IO backend. There is only a
"syscall" backend, unless built for Linux with `#[cfg(feature =
"io_uring")]` which introduces `--io=io-uring`.
Right now, the choice has only been implemented for the CLI. Bindings
and such default to PlatformIO, except when an "in-memory" database has
been chosen.
Can be tested by running CLI with RUST_LOG=debug
Reviewed-by: Preston Thorpe <preston@unlockedlabs.org>
Closes#654
I was planning on starting work on index insertion, but realized we need
to know whether our cursor refers to a table or an index etc., so it
resulted in this refactoring work.
- `cursor_ref` now contains what _type_ of cursor it is (table, index,
pseudo, sorter)
- `program.cursors` is now `program.btree_table_cursors`,
`program.btree_index_cursors` etc and they are unboxed because dynamic
dispatch is no longer necessary
- Cursor trait removed -- 95% of the shit was btree specific anyway, so
I just moved them to `BTreeCursor`. In certain instructions in the VDBE
we expect a btree cursor and in others we expect a pseudo/sorter etc,
lets make that explicit.
- I also removed `BTreeCursor::get_new_rowid()` specific tests that
required macros to generate a mock implementation of the `Cursor` trait
-- main reason is I couldn't figure out how to reimplement this without
the trait, and the second reason is that I don't think we really need
those tests, AND the proc macro is constantly failing in my editor as
well and screwing up `rust-analyzer`
Closes#655
cli: add a new argument to select I/O backend (more than one option only for Linux with io_uring feature).
cli: make both Limbo::new() and Limbo::open_db() use get_io(), unifying parsing of database path and eliminating duplicated code.
`balance_non_root` should be as close as possible to `balance_non_root`
in SQLite. This commits extract `balance_non_root` from `balance` and
renames `balance_leaf` to `balance` as it enables future work on a
complete `balance_non_root` procedure.
Closes#634
`balance_non_root` should be as close as possible to `balance_non_root`
in SQLite. This commits extract `balance_non_root` from `balance` and
renames `balance_leaf` to `balance` as it enables future work on a
complete `balance_non_root` procedure.
This PR fixes the issue with tests running on slow systems such as
github actions, where a couple `datetime` tests would be off by very
small margins and fail (e.g. #626)
Closes#627
TLDR: no need to call either of:
program.emit_insn_with_label_dependency() -> just call program.emit_insn()
program.defer_label_resolution() -> just call program.resolve_label()
Changes:
- make BranchOffset an explicit enum (Label, Offset, Placeholder)
- remove program.emit_insn_with_label_dependency() - label dependency is automatically detected
- for label to offset mapping, use a hashmap from label(negative i32) to offset (positive u32)
- resolve all labels in program.build()
- remove program.defer_label_resolution() - all labels are resolved in build()
I noticed that the parse errors were a bit hard to read - only the
nearest token and the line/col offsets were printed.
I made a first attempt at improving the errors using
[miette](https://github.com/zkat/miette).
- Added derive for `miette::Diagnostic` to both the parser's error type
and LimboError.
- Added miette dependency to both sqlite3_parser and core. The `fancy`
feature is only enabled for the CLI. So the overhead on the libraries
(core, parser) should be minimal.
Some future improvements that can be made further:
- Add spans to AST nodes so that errors can better point to the correct
token. See upstream issue: https://github.com/gwenn/lemon-rs/issues/33
- Construct more errors with offset information. I noticed that most
parser errors are constructed with `None` as the offset.
- The messages are a bit redundant (example "syntax error at (1, 6)").
This can improved.
Comparisons.
Before:
```
❯ cargo run --package limbo --bin limbo database.db --output-mode pretty
...
limbo> selet * from a;
[2025-01-05T11:22:55Z ERROR sqlite3Parser] near "Token([115, 101, 108, 101, 116])": syntax error
Parse error: near "selet": syntax error at (1, 6)
```
<img width="969" alt="image" src="https://github.com/user-
attachments/assets/82651a77-f5ac-4eee-b208-88c6ea7fc9b7" />
After:
```
❯ cargo run --package limbo --bin limbo database.db --output-mode pretty
...
limbo> selet * from a;
[2025-01-05T12:25:52Z ERROR sqlite3Parser] near "Token([115, 101, 108, 101, 116])": syntax error
× near "selet": syntax error at (1, 6)
╭────
1 │ selet * from a
· ▲
· ╰── syntax error
╰────
```
<img width="980" alt="image" src="https://github.com/user-
attachments/assets/747a90e5-5085-41f9-b0fe-25864179ca35" />
Closes#618
This PR adds the `datetime` function, with all the support currently
that date/time have for modifiers, and `julianday` function, as well as
some additional modifiers for date/time/datetime.
There are a couple considerations here, I left a couple comments but
essentially there is going to have to be some more work done to track
the state of the expression during the application of modifiers, to
handle a bunch of edge-cases like re-applying the same timezone modifier
to itself, or converting an integer automatically assumed to be
julianday, into epoch, or `ceiling`/`floor` which will determine
relative addition of time in cases like
```
2024-01-31 +1 month = 2024-03-02
```
which was painful enough to get working to begin with.
I couldn't get the `julianday_converter` library to get the exact same
float precision as sqlite, so function is included that matches their
output, for some reason floating point math + `.floor()` would give the
correct result. They seem to 'round' to 8 decimal places, and I was able
to get this to work with the same output as sqlite, except in cases like
`2234.5`, in which case we return `2234.5000000` because of the `fmt`
precision:
```rust
pub fn exec_julianday(time_value: &OwnedValue) -> Result<String> {
let dt = parse_naive_date_time(time_value);
match dt {
// if we did something heinous like: parse::<f64>().unwrap().to_string()
// that would solve the precision issue, but dear lord...
Some(dt) => Ok(format!("{:.1$}", to_julian_day_exact(&dt), 8)),
None => Ok(String::new()),
}
}
```
Suggestions would be appreciated on the float precision issue.
Reviewed-by: Sonny <14060682+sonhmai@users.noreply.github.com>
Closes#600
- Removed unecessary clones
- Got rid of uneeded scoping in read_complete callback
- Use more rusty syntax e.g. std::cell::RefCell::borrow_mut(&buffer) ->
buffer.borrow_mut()
- Changed write_page.unwrap() to write_page()? to match run_once and
read_page()
No functional change.
Reviewed-by: Jussi Saurio <kacperoza@gmail.com>
Closes#608
I noticed that the parse errors were a bit hard to read - only the nearest token and the line/col offsets were printed.
I made a first attempt at improving the errors using [miette](https://github.com/zkat/miette).
- Added derive for `miette::Diagnostic` to both the parser's error type and LimboError.
- Added miette dependency to both sqlite3_parser and core. The `fancy` feature is only enabled for CLI.
Some future improvements that can be made further:
- Add spans to AST nodes so that errors can better point to the correct token. See upstream issue: https://github.com/gwenn/lemon-rs/issues/33
- Construct more errors with offset information. I noticed that most parser errors are constructed with `None` as the offset.
Comparisons.
Before:
```
❯ cargo run --package limbo --bin limbo database.db --output-mode pretty
...
limbo> selet * from a;
[2025-01-05T11:22:55Z ERROR sqlite3Parser] near "Token([115, 101, 108, 101, 116])": syntax error
Parse error: near "selet": syntax error at (1, 6)
```
After:
```
❯ cargo run --package limbo --bin limbo database.db --output-mode pretty
...
limbo> selet * from a;
[2025-01-05T12:25:52Z ERROR sqlite3Parser] near "Token([115, 101, 108, 101, 116])": syntax error
× near "selet": syntax error at (1, 6)
╭────
1 │ selet * from a
· ▲
· ╰── syntax error
╰────
```
