### Purpose of this PR
- Enhance exception handling logic
- When exceptions has to be thrown from Rust to Java, let's just
return the error message directly.
- Removes JNI call to get error message using
`Java_org_github_tursodatabase_core_LimboDB_getErrorMessageUtf8`
- Add `throwJavaException` to assure that the exception throwing logic
works corretly
Closes#642
## Purpose of this PR
- Implement open function
- Add basic structure for the following
- exception handling
- testing using gradle
## Changes
- Java
- Remove unnecessary example code(Connection.java, Cursor.java,
Limbo.java)
- Implement `open`
- Add exception handling logic
- Add junit test
- Rust
- Add limbo_db.rs which implements native functions defined in
`Limbo.java`
- Remove unnecessary example code in lib.rs
## TODOS
- Implement core features for AbstractDB.java and LimboDB.java (I'm
currently referencing sqlite-java, but there are some minor differences
as we use rust instead of C)
## Reference
- https://github.com/tursodatabase/limbo/issues/615Closes#632
`balance_non_root` should be as close as possible to `balance_non_root`
in SQLite. This commits extract `balance_non_root` from `balance` and
renames `balance_leaf` to `balance` as it enables future work on a
complete `balance_non_root` procedure.
Closes#634
This PR fixes queries like:
```sql
SELECT count(*) FROM users WHERE ((age > 25 OR age < 18) AND (city = 'Boston' OR state = 'MA'));
```
Previously we would return `7516` rows instead of `146`, due to
disregarding the final `AND` from within the nested `OR` conditions,
unconditionally short circuiting when a TRUE is found in an OR
expression without any surrounding relevant context.
or:
```sql
SELECT * FROM users WHERE (((age > 18 AND city = 'New Mario') OR age = 92) AND city = 'Lake Paul');
```
Previously we would incorrectly return the top row:
```
9984|Leah|Russell|..|..|6733 Weber Crossing|New Mario|SC|57707|78
9989|Timothy|Harrison|..|..|782 Wright Harbors|Lake Paul|ID|52330|92
```
Added localized jump targets for OR expressions within AND blocks to
prevent premature short-circuiting and `parent operator` to condition
metadata to trigger them.
If parent operator should instead be another function param on
`translate_conditional_expr`, instead of a field in condition_metadata,
let me know and I can change it.
EDIT: sorry I realize I should have included the other cleanup changes
in a different PR. I will fix this after work
Reviewed-by: Jussi Saurio <kacperoza@gmail.com>
Closes#633
`balance_non_root` should be as close as possible to `balance_non_root`
in SQLite. This commits extract `balance_non_root` from `balance` and
renames `balance_leaf` to `balance` as it enables future work on a
complete `balance_non_root` procedure.
Closes#629
- Fix bug with column being considered rowid alias based on
'primary_key' (non INTEGER pks are not rowid aliases)
- fix logic bug in check_index_scan() that swapped lhs/rhs but not the
comparison op
- Rename eliminate_between to rewrite_exprs and add true/false->1/0 case
there
Closes#630
This PR fixes the issue with tests running on slow systems such as
github actions, where a couple `datetime` tests would be off by very
small margins and fail (e.g. #626)
Closes#627
As was suggested in #502 I have renamed the IO backends from `darwin` to
`unix` and from `linux` to `io_uring`. The `unix` backend is now
available to Linux platforms without io_uring.
On the topic of letting platforms disable io_uring support, the primary
example is Google, which posted this
[blog](https://security.googleblog.com/2023/06/learnings-from-kctf-
vrps-42-linux.html) in 2023 about disabling io_uring by default in most
of their servers/platforms, including Android (prime candidate user for
a SQLite rewrite).
Closes#628
TLDR: no need to call either of:
---
program.emit_insn_with_label_dependency() -> just call
program.emit_insn()
program.defer_label_resolution() -> just call program.resolve_label()
---
Changes:
- make BranchOffset an explicit enum (Label, Offset, Placeholder)
- remove program.emit_insn_with_label_dependency() - label dependency is
automatically detected
- for label to offset mapping, use a hashmap from label(negative i32) to
offset (positive u32)
- resolve all labels in program.build()
- remove program.defer_label_resolution() - all labels are resolved in
build()
Closes#625
TLDR: no need to call either of:
program.emit_insn_with_label_dependency() -> just call program.emit_insn()
program.defer_label_resolution() -> just call program.resolve_label()
Changes:
- make BranchOffset an explicit enum (Label, Offset, Placeholder)
- remove program.emit_insn_with_label_dependency() - label dependency is automatically detected
- for label to offset mapping, use a hashmap from label(negative i32) to offset (positive u32)
- resolve all labels in program.build()
- remove program.defer_label_resolution() - all labels are resolved in build()
Lots of cleanup still left to do. Draft PR for adding support for OPFS
for WASM build (add support for limbo in browser).
Overall explanation of the architecture: this follows the sqlite wasm
architecture for OPFS.
main <> (limbo-worker.js) limbo (VFS - opfs.js) <> opfs-sync-proxy
The main thread loads limbo-worker.js which bootstraps the opfs.js and
opfs-sync-proxy.js and then launches the limbo-wasm.js.
At that point it can be used with worker.postmessage and
worker.onmessage interactions from the main thread.
The VFS provided by opfs.js provides a sync API by offloading async
operations to opfs-sync-proxy.js. This is done through SharedArrayBuffer
and Atomic.wait() to make the actual async operations appear synchronous
for limbo.
resolves#531Closes#594
I noticed that the parse errors were a bit hard to read - only the
nearest token and the line/col offsets were printed.
I made a first attempt at improving the errors using
[miette](https://github.com/zkat/miette).
- Added derive for `miette::Diagnostic` to both the parser's error type
and LimboError.
- Added miette dependency to both sqlite3_parser and core. The `fancy`
feature is only enabled for the CLI. So the overhead on the libraries
(core, parser) should be minimal.
Some future improvements that can be made further:
- Add spans to AST nodes so that errors can better point to the correct
token. See upstream issue: https://github.com/gwenn/lemon-rs/issues/33
- Construct more errors with offset information. I noticed that most
parser errors are constructed with `None` as the offset.
- The messages are a bit redundant (example "syntax error at (1, 6)").
This can improved.
Comparisons.
Before:
```
❯ cargo run --package limbo --bin limbo database.db --output-mode pretty
...
limbo> selet * from a;
[2025-01-05T11:22:55Z ERROR sqlite3Parser] near "Token([115, 101, 108, 101, 116])": syntax error
Parse error: near "selet": syntax error at (1, 6)
```
<img width="969" alt="image" src="https://github.com/user-
attachments/assets/82651a77-f5ac-4eee-b208-88c6ea7fc9b7" />
After:
```
❯ cargo run --package limbo --bin limbo database.db --output-mode pretty
...
limbo> selet * from a;
[2025-01-05T12:25:52Z ERROR sqlite3Parser] near "Token([115, 101, 108, 101, 116])": syntax error
× near "selet": syntax error at (1, 6)
╭────
1 │ selet * from a
· ▲
· ╰── syntax error
╰────
```
<img width="980" alt="image" src="https://github.com/user-
attachments/assets/747a90e5-5085-41f9-b0fe-25864179ca35" />
Closes#618
#514
Introduces a new feature for lazy evaluation in the
`Statement.raw().iterate()` method and includes related changes in both
the test and implementation files. The most important changes include
adding a test case for lazy evaluation, creating a `RowIterator` struct,
and modifying the `iterate` method to use this new struct.
Everything seems to works fine, but suggestions on code improvement and
test use cases are welcoming.
Closes#527
This PR adds the `datetime` function, with all the support currently
that date/time have for modifiers, and `julianday` function, as well as
some additional modifiers for date/time/datetime.
There are a couple considerations here, I left a couple comments but
essentially there is going to have to be some more work done to track
the state of the expression during the application of modifiers, to
handle a bunch of edge-cases like re-applying the same timezone modifier
to itself, or converting an integer automatically assumed to be
julianday, into epoch, or `ceiling`/`floor` which will determine
relative addition of time in cases like
```
2024-01-31 +1 month = 2024-03-02
```
which was painful enough to get working to begin with.
I couldn't get the `julianday_converter` library to get the exact same
float precision as sqlite, so function is included that matches their
output, for some reason floating point math + `.floor()` would give the
correct result. They seem to 'round' to 8 decimal places, and I was able
to get this to work with the same output as sqlite, except in cases like
`2234.5`, in which case we return `2234.5000000` because of the `fmt`
precision:
```rust
pub fn exec_julianday(time_value: &OwnedValue) -> Result<String> {
let dt = parse_naive_date_time(time_value);
match dt {
// if we did something heinous like: parse::<f64>().unwrap().to_string()
// that would solve the precision issue, but dear lord...
Some(dt) => Ok(format!("{:.1$}", to_julian_day_exact(&dt), 8)),
None => Ok(String::new()),
}
}
```
Suggestions would be appreciated on the float precision issue.
Reviewed-by: Sonny <14060682+sonhmai@users.noreply.github.com>
Closes#600
## Purpose of the PR
- Add minimal template code that provides Limbo features.
## Changes
- Added `DB` which is an interface to DB.
- Added 'LimboDB` which is a thin wrapper around native methods provided
using jni.
## TODO
- Incrementally update the code to support jdbc. Refer to [sqlite-
jdbc](https://github.com/xerial/sqlite-jdbc).
## Reference
- https://github.com/tursodatabase/limbo/issues/615Closes#619
