We had not implemented arithmetic operations for text values. This PR
implements this and aligns the behavior with sqlite3 .
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#605
* Add SQLITE_FLAGS env variable handling to compatibility tests as
SQLite does not handle `-q` flag
* Fix inconsistent SQLITE_NOTFOUND error code and add SQLITE_CANTOPEN
code
* Improve compatibility tests to display errors instead of hanging
indefinitely
Closes#599
I was trying to get limbo to a position where we can run the benchmark [
clickbench](https://github.com/ClickHouse/ClickBench/blob/main/sqlite/be
nchmark.sh) and found that `.import` command was not supported in cli.
This PR adds that support for command `.import` which has the same
parameters as sqlite cli.
Do note that not all options from sqlite `.import` is implemented yet in
this PR.
Reviewed-by: Preston Thorpe <preston@unlockedlabs.org>
Closes#598
Implements the `json_extract` function.
In the meantime, the json path has already been implemented by
@petersooley in https://github.com/tursodatabase/limbo/pull/555 which is
a requirement for `json_extract`.
However, this PR takes a different approach and parses the JSON path
using the JSON grammar, because there are a lot of quirks in how a JSON
`key` can look (see the JSON grammar in the Pest file).
The downside is that it allocates more memory than the current
implementation, but might be easier to maintain in the long run.
I included a lot of tests with some quirky behavior of the
`json_extract` (some of them still need some work). I also noticed that
these changed between sqlite versions (had `SQLite 3.43.2` locally and
`3.45` gave different results). Due to this, I'm not sure how much value
there is in trying to be fully compatible with SQLite. Perhaps the
approach taken by @petersooley solves 99% of use-cases?
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#524
data does not match predicate when using index, e.g: `select id, age
from users where age > 90 limit 1;` will return data with age 90
the reason is that the current index seek directly uses record for
comparison, but the record of the index itself is longer than the record
of the key (because it contains the primary key), so Gt is invalid.
since only single-column indexes are currently supported:
https://github.com/tursodatabase/limbo/pull/350, only the first value of
the record is currently used for comparison.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#593
I have added support for like function with escape i.e like(X,Y,Z) .
There is good opportunity to refactor/cleanup the like operations which
can be done in another PR, as I wanted to keep the changes small .
Closes#568
Fixes#552
In our construct regex function, we were not escaping the required
characters properly which was causing the failure.
Limbo output with this branch
```
limbo> select like('\%A', '\A');
1
limbo> select like('A$%', 'A$');
1
limbo> select like('%a.a', 'aaaa');
0
```
Closes#553
#560
Changes to `translate_expr` function:
* [`core/translate/expr.rs`](diffhunk://#diff-
371865d5d7b8bcaed649413c687492e61e94f21387cd9b2c47d989a033888c8bL1558-
R1560): Changed the `amount` parameter in the `Insn::Copy` instruction
from `1` to `0`.
Enhancements to the testing framework:
* [`testing/scalar-functions.test`](diffhunk://#diff-
a046d58ab24eee8207f0ce3199f8d0a609edcef9c24b8ed7f242f7a60e6c1e61R812-
R815): Added a new test `do_execsql_test_regex` to validate that the
`sqlite_version` function returns a valid output.
* [`testing/tester.tcl`](diffhunk://#diff-
316cca92d85df3f78558cc3e60d7420c1fd19a23ecf2bbea534db93ab08ea3ecR29-
R45): Introduced a new procedure `do_execsql_test_regex` to support
regex-based validation of SQL outputs.
Closes#561
Rewrite `Y BETWEEN X AND Z` as `X <= Y AND Y <= Z`. And due to the
support of this optimization rule, limbo should now be able to execute
the `BETWEEN AND` statement.
Closes#490
Implements [json_array](https://sqlite.org/json1.html#jarray).
As a side quest, this PR also fixes an issue with the `CHAR` function
which didn't work properly if the parameters were non-leaf AST nodes.
The PR is quite big, because as I mentioned in https://github.com/tursod
atabase/limbo/issues/127#issuecomment-2541307979 we had to modify
`OwnedValue::Text` to support a `subtype` parameter, which is what
SQLite does.
Closes#504
This change refactors how PRAGMA statements are handled, introducing a
more organized and extensible structure to simplify adding new PRAGMA
properties in the future.
Previously, only the `cache_size` PRAGMA was supported. With this
update, support for the `journal_mode` PRAGMA has been added.
This PR mainly adds custom logic to check equality in ast expressions.
Not sure if this belongs in the `vendored` parser or not, let me know
and I'll bring it out. Also replaces `Vec` arguments with slice refs
where possible, as well as some clippy warnings in the same `emitter`
file.
I'll write some more tests tomorrow to make sure this is as thorough as
possible.
EDIT: failed test same issue referenced in #484. Marking as draft until
more tests + cases added
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#488
Closes#438
**Fixes:**
- Return the actual primary key instead of the rowid when the primary
key is not a rowid alias
sqlite:
```
limbo> create table foo (pk TEXT PRIMARY KEY, value);
sqlite> insert into foo values ('one', 'payload');
sqlite> insert into foo values ('two', 'payload');
sqlite> select * from foo;
one|payload
two|payload
```
limbo now:
```
limbo> create table foo (pk TEXT PRIMARY KEY, value);
limbo> insert into foo values ('one', 'payload');
limbo> insert into foo values ('two', 'payload');
limbo> select * from foo;
one|payload
two|payload
```
limbo before:
```
limbo> select * from foo;
1|payload
2|payload
```
Then, discovered two issues when running the whole TCL test suite
against a copy of `testing.db` that does not have rowid aliases in the
tables:
- Fix trying to convert a Scan into an IndexSearch when the associated
Expr refers to a different instance of the same table in a self-join
- Fix `scan_loop_body_labels` being pushed to in different parts of the
codegen for Scan/Search nodes resulting in incorrect label resolutions
**Additions:**
- Add a new db `testing/testing_norowidalias.db` that is a carbon copy
of `testing/testing.db` except the `id` columns of `users` and
`products` are not rowid aliases (i.e. they are `INT PRIMARY KEY`
instead of `INTEGER PRIMARY KEY`
- Run all TCL tests against both test databases with `do_execsql_test`
```
(testing/testing.db) Running test: coalesce-from-table
(testing/testing_norowidalias.db) Running test: coalesce-from-table
(testing/testing.db) Running test: coalesce-from-table-column
(testing/testing_norowidalias.db) Running test: coalesce-from-table-column
(testing/testing.db) Running test: coalesce-from-table-multiple-columns
(testing/testing_norowidalias.db) Running test: coalesce-from-table-multiple-columns
(testing/testing.db) Running test: glob-fn
(testing/testing_norowidalias.db) Running test: glob-fn
(testing/testing.db) Running test: where-glob
(testing/testing_norowidalias.db) Running test: where-glob
(testing/testing.db) Running test: where-glob-question-mark
(testing/testing_norowidalias.db) Running test: where-glob-question-mark
```
- Allow running tests against specific db file with
`do_execsql_test_on_specific_db $path`
- Wondering if I should add a new table with e.g. text primary key and
add some db-specific tests on that table...?
Closes#449
This adds the initial collection of tests specifically for `limbo`
shell/cli behavior, to separate the concern from testing the underlying
dbms.
Also adds `Echo` mode, and `quiet` mode cli argument for better
experience piping output of limbo on the command line.
After writing a ton of tests in `tcl`, I realized that there is no
reason at all to write these particular tests in tcl anymore, so I
rewrote them in python :)
Shell tests were added to `make test` to run in CI
Closes#487
