Commit Graph

337 Commits

Author SHA1 Message Date
Jussi Saurio
d41dfd0c5d Merge 'Fix rowid search codegen' from Nikita Sivukhin
This PR fixes a bug when index search used incorrect operator if index
column were the "rhs" in the expression (not "lhs" as usual, e.g.
`SELECT * FROM t WHERE 1 < rowid_alias`)

Reviewed-by: Jussi Saurio (@jussisaurio)

Closes #870
2025-02-03 12:38:04 +02:00
Nikita Sivukhin
10e868bc4b add test for index search with "opposite" operator 2025-02-03 11:23:05 +04:00
Nikita Sivukhin
8d513b229f add simple tcl tests 2025-02-02 19:43:13 +04:00
Pekka Enberg
dbb7d1a6ba Merge 'Pagecount' from Glauber Costa
This PR implements the Pagecount pragma, as well as its associated
bytecode opcode

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #819
2025-02-02 09:32:18 +02:00
Glauber Costa
a3387cfd5f implement the pragma page_count
To do that, we also have to implement the vdbe opcode Pagecount.
2025-02-01 19:39:46 -05:00
Nikita Sivukhin
1bd8b4ef7a pass null_eq flag for instructions generated for expressions (not in the conditions) 2025-02-02 02:51:51 +04:00
Nikita Sivukhin
4a9292f657 add tests for previously broken case 2025-02-02 02:42:06 +04:00
Glauber Costa
96987db6ca implement is and is not where constraints
The main difference between = and != is how null values are handled.
SQLite passes a flag "NULLEQ" to Eq and Ne to disambiguate that.
In the presence of that flag, NULL = NULL.

Some prep work is done to make sure we can pass a flag instead of a
boolean to Eq and Ne. I looked into the bitflags crate but got a bit
scared with the list of dependencies.
2025-01-31 23:01:49 -05:00
Glauber Costa
016b815b59 implement pragma table_info
Both () and = variants covered. It is important to make sure that
the transaction is a read transaction, so we cannot hide all that logic
inside update_pragma, and have to make our decision before that.
2025-01-30 20:00:20 -05:00
Pekka Enberg
c779537f2f Merge 'Strftime compatibility solved' from Pedro Muniz
This PR closes #787. Chrono offers to format the string from an iterator
of Format Items. I created a custom iterator that only allows formatters
specified by sqlite. This approach however does not address the
inefficient way that julianday is calculated. Also, with this
implementation we avoid having to maintain a separate vendored package
for strftime that may become incompatible with Chrono in the future.

Closes #792
2025-01-30 13:30:11 +02:00
Pekka Enberg
e66648beb8 Merge 'Add support for offset in select queries' from Ben Li
#739
Started adding support for `LIMIT...OFFSET...`
- New `OffsetLimit` opcode
- `OFFSET` is now supported for:
    - `SELECT...LIMIT...OFFSET`
    - `SELECT...GROUP BY...LIMIT...OFFSET`
    - `SELECT...ORDER BY...LIMIT...OFFSET`
    - Subqueries for `SELECT` statements
**In progress/todo**
- [x] Testing
- [x] Handle negative offset value
- **(will make in separate PR)** Add support for
`DELETE...LIMIT...OFFSET`
- **(will make in separate PR)** Use `limit + offset` sum register from
`OffsetLimit` to constrain number of records inserted into sorter

Closes #779
2025-01-30 13:29:49 +02:00
Pekka Enberg
5614a7751c Merge 'implement isnull / not null for filter expressions' from Glauber Costa
Allow us to write queries like:
        SELECT name, type, sql FROM sqlite_schema where sql isnull
and
        SELECT name, type, sql FROM sqlite_schema where sql not null

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #829
2025-01-30 13:28:53 +02:00
Glauber Costa
effde1cc04 implement isnull / not null for filter expressions
Allow us to write queries like:

	SELECT name, type, sql FROM sqlite_schema where sql isnull

and

	SELECT name, type, sql FROM sqlite_schema where sql not null
2025-01-29 20:58:04 -05:00
Ihor Andrianov
d66329343b add tests 2025-01-30 02:41:05 +02:00
Pekka Enberg
06edf33878 Merge 'json_patch() function implementation' from Ihor Andrianov
First review #820
The function follows RFC 7386 JSON Merge Patch semantics:
* If the patch is null, the target is replaced with null
* If the patch contains a scalar value, the target is replaced with that
value
* If both target and patch are objects, the patch is recursively applied
* null values in the patch result in property removal from the target

Closes #821
2025-01-29 19:54:12 +02:00
Ihor Andrianov
166d0a7165 add more tests and fixed test for latest sqlite v 2025-01-29 18:05:41 +02:00
Ihor Andrianov
0714ab64af add tests for tricky edge cases 2025-01-29 18:05:40 +02:00
Ihor Andrianov
f164dbdb1e fix case where first patched value applied wins 2025-01-29 18:05:40 +02:00
Ihor Andrianov
8ba43bfc17 add tests for duplicate preservation and first-one-win behavior 2025-01-29 18:05:40 +02:00
Ihor Andrianov
3f7458faef add general tests 2025-01-29 18:05:39 +02:00
Pekka Enberg
b9c89e79c2 testing: Add few TCL tests for vector extensions 2025-01-28 14:24:09 +02:00
Pekka Enberg
e8600fa2a1 Merge branch 'main' into static 2025-01-27 09:49:34 +02:00
ben594
847617df63 Created TCL tests for select queries with offset 2025-01-26 16:40:30 -05:00
pedrocarlo
58cf453d2a strftime compatibility issue solved 2025-01-26 17:01:08 -03:00
Harin
0903b9b019 Implemented JSON valid function 2025-01-26 23:35:47 +05:30
pedrocarlo
a316ab51ac feature: implement strftime function 2025-01-25 16:22:53 -03:00
PThorpe92
c5e60d8e08 Enable only uuid by default, change tests back to account for this 2025-01-21 10:20:01 -05:00
PThorpe92
cc63aac305 Fix tests to account for built-in extensions 2025-01-21 09:32:44 -05:00
Harin
da53cc3821 Added Concat Opcode 2025-01-21 00:29:23 +05:30
Jussi Saurio
ce15ad7d32 Simplify added tests with foreach 2025-01-20 17:30:04 +02:00
Jussi Saurio
f88a4d6ac6 Add jump_if_null to cmp insns to account for either operand being NULL 2025-01-20 16:54:39 +02:00
Pekka Enberg
bda1e4e6ab Merge 'Add support for json_object function' from Jorge Hermo
Relates to #127.  This PR is still in draft and I have a few left things
to do (tests, improve implementation), just opening it so anyone can
track this work meanwhile.

Closes #664
2025-01-20 09:36:56 +02:00
Pekka Enberg
c25d9a1824 Merge 'Implement Not' from Vrishabh
This PR adds support for Not operator and Opcode.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #748
2025-01-20 09:17:45 +02:00
ben594
a54222d12b Created basic tcl tests for changes and total_changes 2025-01-19 20:51:16 -05:00
psvri
e616bd5361 Implement Not 2025-01-20 00:21:23 +05:30
Pekka Enberg
3e28541b53 Merge 'Fix null compare operations not giving null' from Vrishabh
In limbo when we do any compare operations like `Eq, gt, lt, gte, lte`
with nulls , we were actually giving the result as true where as sqlite3
gives null. This is because if we had a null, we were incorrectly going
to conditional branch and not increment program by 1. Also the sqlite
generates `ZeroOrNull` op in these cases
(https://github.com/sqlite/sqlite/blob/version-3.45.3/src/expr.c#L4644)
but we were generating a Integer instruction. The below outputs can give
a clearer picture.
This PR aims to fix this.
sqlite3 output
```
SQLite version 3.48.0
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite> select 8 = null;

sqlite> select 8 > null;

sqlite> explain select 8 > null;
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     6     0                    0
1     Integer        1     1     0                    0
2     Gt             3     4     2                    64
3     ZeroOrNull     2     1     3                    0
4     ResultRow      1     1     0                    0
5     Halt           0     0     0                    0
6     Integer        8     2     0                    0
7     Null           0     3     0                    0
8     Goto           0     1     0                    0
sqlite> explain select 8 = null;
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     6     0                    0
1     Integer        1     1     0                    0
2     Eq             3     4     2                    64
3     ZeroOrNull     2     1     3                    0
4     ResultRow      1     1     0                    0
5     Halt           0     0     0                    0
6     Integer        8     2     0                    0
7     Null           0     3     0                    0
8     Goto           0     1     0                    0
```
Limbo Output
```
Limbo v0.0.12
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database
limbo> select 8 = null;
1
limbo> select 8 > null;
1
limbo> explain select 8 > null;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     8     0                    0   Start at 8
1     Integer            8     2     0                    0   r[2]=8
2     Null               0     3     0                    0   r[3]=NULL
3     Integer            1     1     0                    0   r[1]=1
4     Gt                 2     3     6                    0   if r[2]>r[3] goto 6
5     Integer            0     1     0                    0   r[1]=0
6     ResultRow          1     1     0                    0   output=r[1]
7     Halt               0     0     0                    0
8     Transaction        0     0     0                    0
9     Goto               0     1     0                    0
limbo> explain select 8 = null;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     8     0                    0   Start at 8
1     Integer            8     2     0                    0   r[2]=8
2     Null               0     3     0                    0   r[3]=NULL
3     Integer            1     1     0                    0   r[1]=1
4     Eq                 2     3     6                    0   if r[2]==r[3] goto 6
5     Integer            0     1     0                    0   r[1]=0
6     ResultRow          1     1     0                    0   output=r[1]
7     Halt               0     0     0                    0
8     Transaction        0     0     0                    0
9     Goto               0     1     0                    0
limbo>
```
Limbo Output with this PR
```
Limbo v0.0.12
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database
limbo> select 8 = null;

limbo> select 8 > null;

limbo> explain select 8 > null;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     8     0                    0   Start at 8
1     Integer            8     2     0                    0   r[2]=8
2     Null               0     3     0                    0   r[3]=NULL
3     Integer            1     1     0                    0   r[1]=1
4     Gt                 2     3     6                    0   if r[2]>r[3] goto 6
5     ZeroOrNull         2     1     3                    0   ((r[2]=NULL)|(r[3]=NULL)) ? r[1]=NULL : r[1]=0
6     ResultRow          1     1     0                    0   output=r[1]
7     Halt               0     0     0                    0
8     Transaction        0     0     0                    0
9     Goto               0     1     0                    0
limbo>  explain select 8 = null;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     8     0                    0   Start at 8
1     Integer            8     2     0                    0   r[2]=8
2     Null               0     3     0                    0   r[3]=NULL
3     Integer            1     1     0                    0   r[1]=1
4     Eq                 2     3     6                    0   if r[2]==r[3] goto 6
5     ZeroOrNull         2     1     3                    0   ((r[2]=NULL)|(r[3]=NULL)) ? r[1]=NULL : r[1]=0
6     ResultRow          1     1     0                    0   output=r[1]
7     Halt               0     0     0                    0
8     Transaction        0     0     0                    0
9     Goto               0     1     0                    0
```

Closes #733
2025-01-19 09:09:12 +02:00
Krishna Vishal
fa0503f0ce 1. Changes to extension.py
2. chore: cargo fmt
2025-01-19 04:58:05 +05:30
psvri
554244209d Add Null comparision tests 2025-01-19 00:39:10 +05:30
psvri
b966351e1f Implement IsNot operator 2025-01-18 22:49:09 +05:30
psvri
5a13f0790f Implement is operator 2025-01-18 15:57:26 +05:30
psvri
bfadd30f54 Add compare tests 2025-01-18 15:51:31 +05:30
Jussi Saurio
af039ffa6e Merge 'Initial support for aggregate functions in extensions' from Preston Thorpe
#708
This PR adds basic support for the following API for defining
Aggregates, and changes the existing API for scalars.
```rust
register_extension! {
    scalars: { Double },
    aggregates: { MedianState },
}

#[derive(ScalarDerive)]
struct Double;

impl Scalar for Double {
    fn name(&self) -> &'static str {
        "double"
    }
    fn call(&self, args: &[Value]) -> Value {
        if let Some(arg) = args.first() {
            match arg.value_type() {
                ValueType::Float => {
                    let val = arg.to_float().unwrap();
                    Value::from_float(val * 2.0)
                }
                ValueType::Integer => {
                    let val = arg.to_integer().unwrap();
                    Value::from_integer(val * 2)
                }
                _ => {
                    println!("arg: {:?}", arg);
                    Value::null()
                }
            }
        } else {
            Value::null()
        }
    }
}

#[derive(AggregateDerive)]
struct MedianState;

impl AggFunc for MedianState {
    type State = Vec<f64>;

    fn name(&self) -> &'static str {
        "median"
    }
    fn args(&self) -> i32 { 1 }

    fn step(state: &mut Self::State, args: &[Value]) {
        if let Some(val) = args.first().and_then(Value::to_float) {
            state.push(val);
        }
    }

    fn finalize(state: Self::State) -> Value {
        if state.is_empty() {
            return Value::null();
        }
        let mut sorted = state;
        sorted.sort_by(|a, b| a.partial_cmp(b).unwrap());
        let len = sorted.len();
        if len % 2 == 1 {
            Value::from_float(sorted[len / 2])
        } else {
            let mid1 = sorted[len / 2 - 1];
            let mid2 = sorted[len / 2];
            Value::from_float((mid1 + mid2) / 2.0)
        }
    }
}

```
I know it's a bit more verbose than the previous version, but I think in
the long run this will be more robust, and it matches the aggregate API
of implementing a trait on a struct that you derive the relevant trait
on.
Also this allows for better registration of functions, I think passing
in the struct identifiers just feels much better than the `"func_name"
=> function_ptr`

Closes #721
2025-01-18 11:07:06 +02:00
PThorpe92
fc82461eff Complete percentile extension, enable col+delimeter args 2025-01-17 21:15:09 -05:00
PThorpe92
5dfc3b8787 Create simple extension for testing aggregate functions, add tests 2025-01-17 14:30:12 -05:00
PThorpe92
44374b9e69 Clean up scalar trait remove unnecessary args method 2025-01-17 14:13:57 -05:00
CK-7vn
57274fa40b Correct CLI comment handling to mimic sqlite behavior 2025-01-17 13:59:34 -05:00
psvri
e43271f53b Implement regexp extension 2025-01-16 23:15:59 +05:30
Pekka Enberg
f711d2b7ed cli: Improve .schema command output on errors
Improve `.schema` output on errors by marking them as comments. This
allows you to pipe any `.schema` output to another shell.
2025-01-16 14:25:48 +02:00
Jorge Hermo
fa8eb9549a Merge with main 2025-01-15 22:10:35 +01:00
psvri
845de125db Align MustBeInt logic with sqlite 2025-01-16 00:09:45 +05:30