This PR adds the `datetime` function, with all the support currently
that date/time have for modifiers, and `julianday` function, as well as
some additional modifiers for date/time/datetime.
There are a couple considerations here, I left a couple comments but
essentially there is going to have to be some more work done to track
the state of the expression during the application of modifiers, to
handle a bunch of edge-cases like re-applying the same timezone modifier
to itself, or converting an integer automatically assumed to be
julianday, into epoch, or `ceiling`/`floor` which will determine
relative addition of time in cases like
```
2024-01-31 +1 month = 2024-03-02
```
which was painful enough to get working to begin with.
I couldn't get the `julianday_converter` library to get the exact same
float precision as sqlite, so function is included that matches their
output, for some reason floating point math + `.floor()` would give the
correct result. They seem to 'round' to 8 decimal places, and I was able
to get this to work with the same output as sqlite, except in cases like
`2234.5`, in which case we return `2234.5000000` because of the `fmt`
precision:
```rust
pub fn exec_julianday(time_value: &OwnedValue) -> Result<String> {
let dt = parse_naive_date_time(time_value);
match dt {
// if we did something heinous like: parse::<f64>().unwrap().to_string()
// that would solve the precision issue, but dear lord...
Some(dt) => Ok(format!("{:.1$}", to_julian_day_exact(&dt), 8)),
None => Ok(String::new()),
}
}
```
Suggestions would be appreciated on the float precision issue.
Reviewed-by: Sonny <14060682+sonhmai@users.noreply.github.com>
Closes#600
We had not implemented arithmetic operations for text values. This PR
implements this and aligns the behavior with sqlite3 .
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#605
* Add SQLITE_FLAGS env variable handling to compatibility tests as
SQLite does not handle `-q` flag
* Fix inconsistent SQLITE_NOTFOUND error code and add SQLITE_CANTOPEN
code
* Improve compatibility tests to display errors instead of hanging
indefinitely
Closes#599
I was trying to get limbo to a position where we can run the benchmark [
clickbench](https://github.com/ClickHouse/ClickBench/blob/main/sqlite/be
nchmark.sh) and found that `.import` command was not supported in cli.
This PR adds that support for command `.import` which has the same
parameters as sqlite cli.
Do note that not all options from sqlite `.import` is implemented yet in
this PR.
Reviewed-by: Preston Thorpe <preston@unlockedlabs.org>
Closes#598
Implements the `json_extract` function.
In the meantime, the json path has already been implemented by
@petersooley in https://github.com/tursodatabase/limbo/pull/555 which is
a requirement for `json_extract`.
However, this PR takes a different approach and parses the JSON path
using the JSON grammar, because there are a lot of quirks in how a JSON
`key` can look (see the JSON grammar in the Pest file).
The downside is that it allocates more memory than the current
implementation, but might be easier to maintain in the long run.
I included a lot of tests with some quirky behavior of the
`json_extract` (some of them still need some work). I also noticed that
these changed between sqlite versions (had `SQLite 3.43.2` locally and
`3.45` gave different results). Due to this, I'm not sure how much value
there is in trying to be fully compatible with SQLite. Perhaps the
approach taken by @petersooley solves 99% of use-cases?
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#524
data does not match predicate when using index, e.g: `select id, age
from users where age > 90 limit 1;` will return data with age 90
the reason is that the current index seek directly uses record for
comparison, but the record of the index itself is longer than the record
of the key (because it contains the primary key), so Gt is invalid.
since only single-column indexes are currently supported:
https://github.com/tursodatabase/limbo/pull/350, only the first value of
the record is currently used for comparison.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#593
I have added support for like function with escape i.e like(X,Y,Z) .
There is good opportunity to refactor/cleanup the like operations which
can be done in another PR, as I wanted to keep the changes small .
Closes#568
Fixes#552
In our construct regex function, we were not escaping the required
characters properly which was causing the failure.
Limbo output with this branch
```
limbo> select like('\%A', '\A');
1
limbo> select like('A$%', 'A$');
1
limbo> select like('%a.a', 'aaaa');
0
```
Closes#553
#560
Changes to `translate_expr` function:
* [`core/translate/expr.rs`](diffhunk://#diff-
371865d5d7b8bcaed649413c687492e61e94f21387cd9b2c47d989a033888c8bL1558-
R1560): Changed the `amount` parameter in the `Insn::Copy` instruction
from `1` to `0`.
Enhancements to the testing framework:
* [`testing/scalar-functions.test`](diffhunk://#diff-
a046d58ab24eee8207f0ce3199f8d0a609edcef9c24b8ed7f242f7a60e6c1e61R812-
R815): Added a new test `do_execsql_test_regex` to validate that the
`sqlite_version` function returns a valid output.
* [`testing/tester.tcl`](diffhunk://#diff-
316cca92d85df3f78558cc3e60d7420c1fd19a23ecf2bbea534db93ab08ea3ecR29-
R45): Introduced a new procedure `do_execsql_test_regex` to support
regex-based validation of SQL outputs.
Closes#561
Rewrite `Y BETWEEN X AND Z` as `X <= Y AND Y <= Z`. And due to the
support of this optimization rule, limbo should now be able to execute
the `BETWEEN AND` statement.
Closes#490
