Commit Graph

287 Commits

Author SHA1 Message Date
psvri
845de125db Align MustBeInt logic with sqlite 2025-01-16 00:09:45 +05:30
Pekka Enberg
bdc06f2d66 Merge 'Implement ShiftRight' from Vrishabh
This PR adds support for ShiftRight operator and Opcode.

Closes #703
2025-01-15 18:53:23 +02:00
Pekka Enberg
c01c80baee Merge 'Run all statements from sql argument in cli' from Vrishabh
When we had multiple sql statements in cli/interactive shell, we were
only running one from them as  shown below. This PR fixes them.
One added benefit of this is we can add complex sql in the tcl test
files like the changes in insert.test .
sqlite3 output
```
❯ sqlite3  :memory: "select 1;select 2"
1
2
```
Limbo main branch output
```
❯ ./target/debug/limbo.exe :memory: "select 1;select 2;"
1
```
Limbos output with this PR
```
❯ ./target/debug/limbo.exe :memory: "select 1;select 2;"
1
2
```

Reviewed-by: Preston Thorpe <cory.pride83@gmail.com>

Closes #673
2025-01-15 18:44:17 +02:00
Pekka Enberg
7c549bc978 Merge 'Expr: fix recursive binary operation logic' from Jussi Saurio
I believe this closes #682
```
limbo> CREATE TABLE proficient_barrett (imaginative_etrebilal BLOB,lovely_wilson BLOB);
INSERT INTO proficient_barrett VALUES (X'656E67726F7373696E675F636861636F', X'776F6E64726F75735F626F75726E65');
limbo> SELECT * FROM proficient_barrett
WHERE (
    (
        (
            (
                imaginative_etrebilal != X'6661766F7261626C655F636F726573'
                OR
                (imaginative_etrebilal > X'656E67726F7373696E675F6368616439')
            )
            AND
            (
                imaginative_etrebilal = X'656E676167696E675F6E6163696F6E616C'
                OR
                TRUE
            )
        )
        OR
        FALSE
    )
    AND
    (
        imaginative_etrebilal > X'656E67726F7373696E675F63686164F6'
        OR
        TRUE
    )
);
engrossing_chaco|wondrous_bourne
```
@PThorpe92 I don't think we need the `parent_op` machinery at all, we
just need to not jump to the `jump_target_when_true` label given by the
parent if we are evaluating the first condition of an AND.
related: https://github.com/tursodatabase/limbo/pull/633

Reviewed-by: Preston Thorpe <cory.pride83@gmail.com>

Closes #698
2025-01-15 18:32:59 +02:00
psvri
d3f28c51f4 Implement ShiftRight 2025-01-15 21:21:51 +05:30
psvri
5b4d82abbf Implement ShiftLeft 2025-01-15 18:54:07 +05:30
psvri
9cc9577c91 Run all statements from sql argument in cli 2025-01-15 18:19:39 +05:30
Jussi Saurio
f8b3b06163 Expr: fix recursive binary operation logic 2025-01-15 14:12:08 +02:00
Kould
1bf651bd37 chore: rollback using rowid(sqlite3 unsupported) 2025-01-14 22:56:49 +08:00
Kould
5305a9d0fd feat: support keyword rowid 2025-01-14 22:41:40 +08:00
PThorpe92
9c208dc866 Add tests for first extension 2025-01-14 07:27:35 -05:00
Pekka Enberg
1e94dbffcc Merge branch 'main' into json-error-position 2025-01-13 18:21:37 +02:00
Jussi Saurio
3e2993f5f5 Merge pull request #626 from psvri/fix_csv_quote_import
Fix import csv failing when file contains single quote
2025-01-11 13:26:45 +02:00
psvri
ce8600a695 Fix import csv failing when single quote is in string 2025-01-11 13:48:21 +05:30
Peter Sooley
b5fed15997 implement json_error_position 2025-01-10 11:12:30 -08:00
Kacper Madej
002d2e3dde Uncomment failing tests 2025-01-10 19:26:39 +07:00
Kacper Madej
7efa87ff31 Merge branch 'main' into right-arrow-json 2025-01-10 19:25:56 +07:00
Jussi Saurio
1bcdf99eab core/optimizer: do expression rewriting on all expressions 2025-01-10 10:04:07 +02:00
Kacper Madej
91d4ac3ac0 Fix expression chaining 2025-01-10 12:14:57 +07:00
Kacper Madej
0f4e5ad26d Implement simplified json path 2025-01-10 11:50:43 +07:00
Kacper Madej
743a8b2d94 Merge branch 'main' into right-arrow-json 2025-01-10 11:28:13 +07:00
Pekka Enberg
317acb842b Merge 'Implement json_type' from Kacper Madej
Closes #631
2025-01-09 14:25:32 +02:00
Kacper Madej
f4e331231b More tests 2025-01-09 17:22:19 +07:00
Kacper Madej
74e19e2148 Optimization 2025-01-09 17:16:58 +07:00
Kacper Madej
dd533414ef Implement -> and ->> operators for json 2025-01-09 15:38:32 +07:00
Kacper Madej
eebf9bfaac Implement json_type 2025-01-09 11:29:17 +07:00
PThorpe92
183797898b Add tests for nested conditional expressions 2025-01-08 17:19:37 -05:00
Jussi Saurio
925bd62cbc fix logic bug in check_index_scan() that swapped lhs/rhs but not the comparison op 2025-01-08 08:20:13 +02:00
Pekka Enberg
ba28999d05 Merge 'Add partial support for datetime() function' from Preston Thorpe
This PR adds the `datetime` function, with all the support currently
that date/time have for modifiers, and `julianday` function, as well as
some additional modifiers for date/time/datetime.
There are a couple considerations here, I left a couple comments but
essentially there is going to have to be some more work done to track
the state of the expression during the application of modifiers, to
handle a bunch of edge-cases like re-applying the same timezone modifier
to itself, or converting an integer automatically assumed to be
julianday, into epoch, or `ceiling`/`floor` which will determine
relative addition of time in cases like
```
2024-01-31 +1 month = 2024-03-02
```
which was painful enough to get working to begin with.
I couldn't get the `julianday_converter` library to get the exact same
float precision as sqlite, so function is included that matches their
output, for some reason floating point math + `.floor()` would give the
correct result. They seem to 'round' to 8 decimal places, and I was able
to get this to work with the same output as sqlite, except in cases like
`2234.5`, in which case we return `2234.5000000` because of the `fmt`
precision:
```rust
pub fn exec_julianday(time_value: &OwnedValue) -> Result<String> {
    let dt = parse_naive_date_time(time_value);
    match dt {
        // if we did something heinous like: parse::<f64>().unwrap().to_string()
        // that would solve the precision issue, but dear lord...
        Some(dt) => Ok(format!("{:.1$}", to_julian_day_exact(&dt), 8)),
        None => Ok(String::new()),
    }
}
```
Suggestions would be appreciated on the float precision issue.

Reviewed-by: Sonny <14060682+sonhmai@users.noreply.github.com>

Closes #600
2025-01-05 20:13:13 +02:00
psvri
2d84956fda Fix quote escape in literals 2025-01-05 01:35:29 +05:30
PThorpe92
ca428b3dda Julianday function and additional tests/comments 2025-01-04 10:42:34 -05:00
PThorpe92
9a635be7b8 Add tests for new modifiers and datetime func 2025-01-04 08:41:05 -05:00
psvri
18137c932e Fix integer overflow output to be same result as sqlite3 2025-01-04 18:14:09 +05:30
Pekka Enberg
fc60e544af Merge 'Fix arithmetic operations for text values' from Vrishabh
We had not implemented arithmetic operations for text values. This PR
implements this and aligns the behavior with sqlite3 .

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #605
2025-01-04 13:40:03 +02:00
Pekka Enberg
b5e95d7b5d Merge 'Remove SQLite headers dependency from compat tests' from dkaluza
* Add SQLITE_FLAGS env variable handling to compatibility tests as
SQLite does not handle `-q` flag
* Fix inconsistent SQLITE_NOTFOUND error code and add SQLITE_CANTOPEN
code
* Improve compatibility tests to display errors instead of hanging
indefinitely

Closes #599
2025-01-04 10:12:28 +02:00
Pekka Enberg
3ddbb4bdea Merge 'Add CSV import support to shell' from Vrishabh
I was trying to get limbo to a position where we can run the benchmark [
clickbench](https://github.com/ClickHouse/ClickBench/blob/main/sqlite/be
nchmark.sh)  and found that `.import` command was not supported in cli.
This PR adds that support for command `.import` which has the same
parameters as sqlite cli.
Do note that not all options from sqlite `.import` is implemented yet in
this PR.

Reviewed-by: Preston Thorpe <preston@unlockedlabs.org>

Closes #598
2025-01-04 10:11:41 +02:00
Jussi Saurio
a934ead904 Merge 'Json extract' from Kacper Madej
Implements the `json_extract` function.
In the meantime, the json path has already been implemented by
@petersooley in https://github.com/tursodatabase/limbo/pull/555 which is
a requirement for `json_extract`.
However, this PR takes a different approach and parses the JSON path
using the JSON grammar, because there are a lot of quirks in how a JSON
`key` can look (see the JSON grammar in the Pest file).
The downside is that it allocates more memory than the current
implementation, but might be easier to maintain in the long run.
I included a lot of tests with some quirky behavior of the
`json_extract` (some of them still need some work). I also noticed that
these changed between sqlite versions (had `SQLite 3.43.2` locally and
`3.45` gave different results). Due to this, I'm not sure how much value
there is in trying to be fully compatible with SQLite. Perhaps the
approach taken by @petersooley solves 99% of use-cases?

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #524
2025-01-03 23:53:29 +02:00
Jussi Saurio
0fefffbbcb Merge 'fix: index seek wrong on SeekOp::LT\SeekOp::GT' from Kould
data does not match predicate when using index, e.g: `select id, age
from users where age > 90 limit 1;` will return data with age  90
the reason is that the current index seek directly uses record for
comparison, but the record of the index itself is longer than the record
of the key (because it contains the primary key), so Gt is invalid.
since only single-column indexes are currently supported:
https://github.com/tursodatabase/limbo/pull/350, only the first value of
the record is currently used for comparison.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #593
2025-01-03 23:27:24 +02:00
psvri
2b879a4f40 Fix arithmetic operations for text values 2025-01-04 00:34:04 +05:30
Daniel Kaluza
9ab0e90807 Fix compatibility tests not showing errors and hanging indefinitely in case of error 2025-01-03 11:19:21 +01:00
psvri
1f21cf6a71 Feat: Import csv support 2025-01-03 15:20:22 +05:30
Kacper Madej
f27f873804 PR remarks #2 2025-01-02 15:09:16 +07:00
Kacper Madej
7d7d202ffe PR remarks 2025-01-02 14:41:47 +07:00
Kould
e5d0ad044e fix: index seek wrong 2025-01-02 11:11:44 +08:00
Jussi Saurio
2066475e03 feat: subqueries in FROM clause 2024-12-31 14:18:29 +02:00
Kacper Madej
719eda7cf7 Mark tests to fix for newer SQLite version 2024-12-31 15:56:35 +07:00
Kacper Madej
692301e72c Merge branch 'main' into json-extract 2024-12-31 15:53:08 +07:00
Kacper Madej
2e730ead25 Merge branch 'main' into json-extract 2024-12-31 15:41:18 +07:00
Kacper Madej
ad9acf7400 Add support for json_extract 2024-12-31 15:11:36 +07:00
psvri
3ac3fdf0a2 Fix glob 2024-12-30 17:02:31 +05:30