Commit Graph

306 Commits

Author SHA1 Message Date
Pekka Enberg
bda1e4e6ab Merge 'Add support for json_object function' from Jorge Hermo
Relates to #127.  This PR is still in draft and I have a few left things
to do (tests, improve implementation), just opening it so anyone can
track this work meanwhile.

Closes #664
2025-01-20 09:36:56 +02:00
Pekka Enberg
c25d9a1824 Merge 'Implement Not' from Vrishabh
This PR adds support for Not operator and Opcode.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #748
2025-01-20 09:17:45 +02:00
ben594
a54222d12b Created basic tcl tests for changes and total_changes 2025-01-19 20:51:16 -05:00
psvri
e616bd5361 Implement Not 2025-01-20 00:21:23 +05:30
Pekka Enberg
3e28541b53 Merge 'Fix null compare operations not giving null' from Vrishabh
In limbo when we do any compare operations like `Eq, gt, lt, gte, lte`
with nulls , we were actually giving the result as true where as sqlite3
gives null. This is because if we had a null, we were incorrectly going
to conditional branch and not increment program by 1. Also the sqlite
generates `ZeroOrNull` op in these cases
(https://github.com/sqlite/sqlite/blob/version-3.45.3/src/expr.c#L4644)
but we were generating a Integer instruction. The below outputs can give
a clearer picture.
This PR aims to fix this.
sqlite3 output
```
SQLite version 3.48.0
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite> select 8 = null;

sqlite> select 8 > null;

sqlite> explain select 8 > null;
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     6     0                    0
1     Integer        1     1     0                    0
2     Gt             3     4     2                    64
3     ZeroOrNull     2     1     3                    0
4     ResultRow      1     1     0                    0
5     Halt           0     0     0                    0
6     Integer        8     2     0                    0
7     Null           0     3     0                    0
8     Goto           0     1     0                    0
sqlite> explain select 8 = null;
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     6     0                    0
1     Integer        1     1     0                    0
2     Eq             3     4     2                    64
3     ZeroOrNull     2     1     3                    0
4     ResultRow      1     1     0                    0
5     Halt           0     0     0                    0
6     Integer        8     2     0                    0
7     Null           0     3     0                    0
8     Goto           0     1     0                    0
```
Limbo Output
```
Limbo v0.0.12
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database
limbo> select 8 = null;
1
limbo> select 8 > null;
1
limbo> explain select 8 > null;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     8     0                    0   Start at 8
1     Integer            8     2     0                    0   r[2]=8
2     Null               0     3     0                    0   r[3]=NULL
3     Integer            1     1     0                    0   r[1]=1
4     Gt                 2     3     6                    0   if r[2]>r[3] goto 6
5     Integer            0     1     0                    0   r[1]=0
6     ResultRow          1     1     0                    0   output=r[1]
7     Halt               0     0     0                    0
8     Transaction        0     0     0                    0
9     Goto               0     1     0                    0
limbo> explain select 8 = null;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     8     0                    0   Start at 8
1     Integer            8     2     0                    0   r[2]=8
2     Null               0     3     0                    0   r[3]=NULL
3     Integer            1     1     0                    0   r[1]=1
4     Eq                 2     3     6                    0   if r[2]==r[3] goto 6
5     Integer            0     1     0                    0   r[1]=0
6     ResultRow          1     1     0                    0   output=r[1]
7     Halt               0     0     0                    0
8     Transaction        0     0     0                    0
9     Goto               0     1     0                    0
limbo>
```
Limbo Output with this PR
```
Limbo v0.0.12
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database
limbo> select 8 = null;

limbo> select 8 > null;

limbo> explain select 8 > null;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     8     0                    0   Start at 8
1     Integer            8     2     0                    0   r[2]=8
2     Null               0     3     0                    0   r[3]=NULL
3     Integer            1     1     0                    0   r[1]=1
4     Gt                 2     3     6                    0   if r[2]>r[3] goto 6
5     ZeroOrNull         2     1     3                    0   ((r[2]=NULL)|(r[3]=NULL)) ? r[1]=NULL : r[1]=0
6     ResultRow          1     1     0                    0   output=r[1]
7     Halt               0     0     0                    0
8     Transaction        0     0     0                    0
9     Goto               0     1     0                    0
limbo>  explain select 8 = null;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     8     0                    0   Start at 8
1     Integer            8     2     0                    0   r[2]=8
2     Null               0     3     0                    0   r[3]=NULL
3     Integer            1     1     0                    0   r[1]=1
4     Eq                 2     3     6                    0   if r[2]==r[3] goto 6
5     ZeroOrNull         2     1     3                    0   ((r[2]=NULL)|(r[3]=NULL)) ? r[1]=NULL : r[1]=0
6     ResultRow          1     1     0                    0   output=r[1]
7     Halt               0     0     0                    0
8     Transaction        0     0     0                    0
9     Goto               0     1     0                    0
```

Closes #733
2025-01-19 09:09:12 +02:00
Krishna Vishal
fa0503f0ce 1. Changes to extension.py
2. chore: cargo fmt
2025-01-19 04:58:05 +05:30
psvri
554244209d Add Null comparision tests 2025-01-19 00:39:10 +05:30
psvri
b966351e1f Implement IsNot operator 2025-01-18 22:49:09 +05:30
psvri
5a13f0790f Implement is operator 2025-01-18 15:57:26 +05:30
psvri
bfadd30f54 Add compare tests 2025-01-18 15:51:31 +05:30
Jussi Saurio
af039ffa6e Merge 'Initial support for aggregate functions in extensions' from Preston Thorpe
#708
This PR adds basic support for the following API for defining
Aggregates, and changes the existing API for scalars.
```rust
register_extension! {
    scalars: { Double },
    aggregates: { MedianState },
}

#[derive(ScalarDerive)]
struct Double;

impl Scalar for Double {
    fn name(&self) -> &'static str {
        "double"
    }
    fn call(&self, args: &[Value]) -> Value {
        if let Some(arg) = args.first() {
            match arg.value_type() {
                ValueType::Float => {
                    let val = arg.to_float().unwrap();
                    Value::from_float(val * 2.0)
                }
                ValueType::Integer => {
                    let val = arg.to_integer().unwrap();
                    Value::from_integer(val * 2)
                }
                _ => {
                    println!("arg: {:?}", arg);
                    Value::null()
                }
            }
        } else {
            Value::null()
        }
    }
}

#[derive(AggregateDerive)]
struct MedianState;

impl AggFunc for MedianState {
    type State = Vec<f64>;

    fn name(&self) -> &'static str {
        "median"
    }
    fn args(&self) -> i32 { 1 }

    fn step(state: &mut Self::State, args: &[Value]) {
        if let Some(val) = args.first().and_then(Value::to_float) {
            state.push(val);
        }
    }

    fn finalize(state: Self::State) -> Value {
        if state.is_empty() {
            return Value::null();
        }
        let mut sorted = state;
        sorted.sort_by(|a, b| a.partial_cmp(b).unwrap());
        let len = sorted.len();
        if len % 2 == 1 {
            Value::from_float(sorted[len / 2])
        } else {
            let mid1 = sorted[len / 2 - 1];
            let mid2 = sorted[len / 2];
            Value::from_float((mid1 + mid2) / 2.0)
        }
    }
}

```
I know it's a bit more verbose than the previous version, but I think in
the long run this will be more robust, and it matches the aggregate API
of implementing a trait on a struct that you derive the relevant trait
on.
Also this allows for better registration of functions, I think passing
in the struct identifiers just feels much better than the `"func_name"
=> function_ptr`

Closes #721
2025-01-18 11:07:06 +02:00
PThorpe92
fc82461eff Complete percentile extension, enable col+delimeter args 2025-01-17 21:15:09 -05:00
PThorpe92
5dfc3b8787 Create simple extension for testing aggregate functions, add tests 2025-01-17 14:30:12 -05:00
PThorpe92
44374b9e69 Clean up scalar trait remove unnecessary args method 2025-01-17 14:13:57 -05:00
CK-7vn
57274fa40b Correct CLI comment handling to mimic sqlite behavior 2025-01-17 13:59:34 -05:00
psvri
e43271f53b Implement regexp extension 2025-01-16 23:15:59 +05:30
Pekka Enberg
f711d2b7ed cli: Improve .schema command output on errors
Improve `.schema` output on errors by marking them as comments. This
allows you to pipe any `.schema` output to another shell.
2025-01-16 14:25:48 +02:00
Jorge Hermo
fa8eb9549a Merge with main 2025-01-15 22:10:35 +01:00
psvri
845de125db Align MustBeInt logic with sqlite 2025-01-16 00:09:45 +05:30
Pekka Enberg
bdc06f2d66 Merge 'Implement ShiftRight' from Vrishabh
This PR adds support for ShiftRight operator and Opcode.

Closes #703
2025-01-15 18:53:23 +02:00
Pekka Enberg
c01c80baee Merge 'Run all statements from sql argument in cli' from Vrishabh
When we had multiple sql statements in cli/interactive shell, we were
only running one from them as  shown below. This PR fixes them.
One added benefit of this is we can add complex sql in the tcl test
files like the changes in insert.test .
sqlite3 output
```
❯ sqlite3  :memory: "select 1;select 2"
1
2
```
Limbo main branch output
```
❯ ./target/debug/limbo.exe :memory: "select 1;select 2;"
1
```
Limbos output with this PR
```
❯ ./target/debug/limbo.exe :memory: "select 1;select 2;"
1
2
```

Reviewed-by: Preston Thorpe <cory.pride83@gmail.com>

Closes #673
2025-01-15 18:44:17 +02:00
Pekka Enberg
7c549bc978 Merge 'Expr: fix recursive binary operation logic' from Jussi Saurio
I believe this closes #682
```
limbo> CREATE TABLE proficient_barrett (imaginative_etrebilal BLOB,lovely_wilson BLOB);
INSERT INTO proficient_barrett VALUES (X'656E67726F7373696E675F636861636F', X'776F6E64726F75735F626F75726E65');
limbo> SELECT * FROM proficient_barrett
WHERE (
    (
        (
            (
                imaginative_etrebilal != X'6661766F7261626C655F636F726573'
                OR
                (imaginative_etrebilal > X'656E67726F7373696E675F6368616439')
            )
            AND
            (
                imaginative_etrebilal = X'656E676167696E675F6E6163696F6E616C'
                OR
                TRUE
            )
        )
        OR
        FALSE
    )
    AND
    (
        imaginative_etrebilal > X'656E67726F7373696E675F63686164F6'
        OR
        TRUE
    )
);
engrossing_chaco|wondrous_bourne
```
@PThorpe92 I don't think we need the `parent_op` machinery at all, we
just need to not jump to the `jump_target_when_true` label given by the
parent if we are evaluating the first condition of an AND.
related: https://github.com/tursodatabase/limbo/pull/633

Reviewed-by: Preston Thorpe <cory.pride83@gmail.com>

Closes #698
2025-01-15 18:32:59 +02:00
psvri
d3f28c51f4 Implement ShiftRight 2025-01-15 21:21:51 +05:30
psvri
5b4d82abbf Implement ShiftLeft 2025-01-15 18:54:07 +05:30
psvri
9cc9577c91 Run all statements from sql argument in cli 2025-01-15 18:19:39 +05:30
Jussi Saurio
f8b3b06163 Expr: fix recursive binary operation logic 2025-01-15 14:12:08 +02:00
Kould
1bf651bd37 chore: rollback using rowid(sqlite3 unsupported) 2025-01-14 22:56:49 +08:00
Kould
5305a9d0fd feat: support keyword rowid 2025-01-14 22:41:40 +08:00
PThorpe92
9c208dc866 Add tests for first extension 2025-01-14 07:27:35 -05:00
Jorge Hermo
fea8d19359 test: add TCL tests for json_object 2025-01-14 00:00:31 +01:00
Pekka Enberg
1e94dbffcc Merge branch 'main' into json-error-position 2025-01-13 18:21:37 +02:00
Jussi Saurio
3e2993f5f5 Merge pull request #626 from psvri/fix_csv_quote_import
Fix import csv failing when file contains single quote
2025-01-11 13:26:45 +02:00
psvri
ce8600a695 Fix import csv failing when single quote is in string 2025-01-11 13:48:21 +05:30
Peter Sooley
b5fed15997 implement json_error_position 2025-01-10 11:12:30 -08:00
Kacper Madej
002d2e3dde Uncomment failing tests 2025-01-10 19:26:39 +07:00
Kacper Madej
7efa87ff31 Merge branch 'main' into right-arrow-json 2025-01-10 19:25:56 +07:00
Jussi Saurio
1bcdf99eab core/optimizer: do expression rewriting on all expressions 2025-01-10 10:04:07 +02:00
Kacper Madej
91d4ac3ac0 Fix expression chaining 2025-01-10 12:14:57 +07:00
Kacper Madej
0f4e5ad26d Implement simplified json path 2025-01-10 11:50:43 +07:00
Kacper Madej
743a8b2d94 Merge branch 'main' into right-arrow-json 2025-01-10 11:28:13 +07:00
Pekka Enberg
317acb842b Merge 'Implement json_type' from Kacper Madej
Closes #631
2025-01-09 14:25:32 +02:00
Kacper Madej
f4e331231b More tests 2025-01-09 17:22:19 +07:00
Kacper Madej
74e19e2148 Optimization 2025-01-09 17:16:58 +07:00
Kacper Madej
dd533414ef Implement -> and ->> operators for json 2025-01-09 15:38:32 +07:00
Kacper Madej
eebf9bfaac Implement json_type 2025-01-09 11:29:17 +07:00
PThorpe92
183797898b Add tests for nested conditional expressions 2025-01-08 17:19:37 -05:00
Jussi Saurio
925bd62cbc fix logic bug in check_index_scan() that swapped lhs/rhs but not the comparison op 2025-01-08 08:20:13 +02:00
Pekka Enberg
ba28999d05 Merge 'Add partial support for datetime() function' from Preston Thorpe
This PR adds the `datetime` function, with all the support currently
that date/time have for modifiers, and `julianday` function, as well as
some additional modifiers for date/time/datetime.
There are a couple considerations here, I left a couple comments but
essentially there is going to have to be some more work done to track
the state of the expression during the application of modifiers, to
handle a bunch of edge-cases like re-applying the same timezone modifier
to itself, or converting an integer automatically assumed to be
julianday, into epoch, or `ceiling`/`floor` which will determine
relative addition of time in cases like
```
2024-01-31 +1 month = 2024-03-02
```
which was painful enough to get working to begin with.
I couldn't get the `julianday_converter` library to get the exact same
float precision as sqlite, so function is included that matches their
output, for some reason floating point math + `.floor()` would give the
correct result. They seem to 'round' to 8 decimal places, and I was able
to get this to work with the same output as sqlite, except in cases like
`2234.5`, in which case we return `2234.5000000` because of the `fmt`
precision:
```rust
pub fn exec_julianday(time_value: &OwnedValue) -> Result<String> {
    let dt = parse_naive_date_time(time_value);
    match dt {
        // if we did something heinous like: parse::<f64>().unwrap().to_string()
        // that would solve the precision issue, but dear lord...
        Some(dt) => Ok(format!("{:.1$}", to_julian_day_exact(&dt), 8)),
        None => Ok(String::new()),
    }
}
```
Suggestions would be appreciated on the float precision issue.

Reviewed-by: Sonny <14060682+sonhmai@users.noreply.github.com>

Closes #600
2025-01-05 20:13:13 +02:00
psvri
2d84956fda Fix quote escape in literals 2025-01-05 01:35:29 +05:30
PThorpe92
ca428b3dda Julianday function and additional tests/comments 2025-01-04 10:42:34 -05:00