Commit Graph

290 Commits

Author SHA1 Message Date
psvri
e616bd5361 Implement Not 2025-01-20 00:21:23 +05:30
Pekka Enberg
f5e5428d45 Merge 'Syntactic improvements' from Jorge López Tello
This is a purely syntactic PR. It doesn't change behavior, just rewrites
some loops and removes unneeded parts, like lifetime annotations and
references. Mainly because the Clippy and IDE warnings get annoying.
Don't worry about the number of commits, I just separated based on type
of change.

Closes #732
2025-01-19 12:17:28 +02:00
Pekka Enberg
3e28541b53 Merge 'Fix null compare operations not giving null' from Vrishabh
In limbo when we do any compare operations like `Eq, gt, lt, gte, lte`
with nulls , we were actually giving the result as true where as sqlite3
gives null. This is because if we had a null, we were incorrectly going
to conditional branch and not increment program by 1. Also the sqlite
generates `ZeroOrNull` op in these cases
(https://github.com/sqlite/sqlite/blob/version-3.45.3/src/expr.c#L4644)
but we were generating a Integer instruction. The below outputs can give
a clearer picture.
This PR aims to fix this.
sqlite3 output
```
SQLite version 3.48.0
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite> select 8 = null;

sqlite> select 8 > null;

sqlite> explain select 8 > null;
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     6     0                    0
1     Integer        1     1     0                    0
2     Gt             3     4     2                    64
3     ZeroOrNull     2     1     3                    0
4     ResultRow      1     1     0                    0
5     Halt           0     0     0                    0
6     Integer        8     2     0                    0
7     Null           0     3     0                    0
8     Goto           0     1     0                    0
sqlite> explain select 8 = null;
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     6     0                    0
1     Integer        1     1     0                    0
2     Eq             3     4     2                    64
3     ZeroOrNull     2     1     3                    0
4     ResultRow      1     1     0                    0
5     Halt           0     0     0                    0
6     Integer        8     2     0                    0
7     Null           0     3     0                    0
8     Goto           0     1     0                    0
```
Limbo Output
```
Limbo v0.0.12
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database
limbo> select 8 = null;
1
limbo> select 8 > null;
1
limbo> explain select 8 > null;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     8     0                    0   Start at 8
1     Integer            8     2     0                    0   r[2]=8
2     Null               0     3     0                    0   r[3]=NULL
3     Integer            1     1     0                    0   r[1]=1
4     Gt                 2     3     6                    0   if r[2]>r[3] goto 6
5     Integer            0     1     0                    0   r[1]=0
6     ResultRow          1     1     0                    0   output=r[1]
7     Halt               0     0     0                    0
8     Transaction        0     0     0                    0
9     Goto               0     1     0                    0
limbo> explain select 8 = null;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     8     0                    0   Start at 8
1     Integer            8     2     0                    0   r[2]=8
2     Null               0     3     0                    0   r[3]=NULL
3     Integer            1     1     0                    0   r[1]=1
4     Eq                 2     3     6                    0   if r[2]==r[3] goto 6
5     Integer            0     1     0                    0   r[1]=0
6     ResultRow          1     1     0                    0   output=r[1]
7     Halt               0     0     0                    0
8     Transaction        0     0     0                    0
9     Goto               0     1     0                    0
limbo>
```
Limbo Output with this PR
```
Limbo v0.0.12
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database
limbo> select 8 = null;

limbo> select 8 > null;

limbo> explain select 8 > null;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     8     0                    0   Start at 8
1     Integer            8     2     0                    0   r[2]=8
2     Null               0     3     0                    0   r[3]=NULL
3     Integer            1     1     0                    0   r[1]=1
4     Gt                 2     3     6                    0   if r[2]>r[3] goto 6
5     ZeroOrNull         2     1     3                    0   ((r[2]=NULL)|(r[3]=NULL)) ? r[1]=NULL : r[1]=0
6     ResultRow          1     1     0                    0   output=r[1]
7     Halt               0     0     0                    0
8     Transaction        0     0     0                    0
9     Goto               0     1     0                    0
limbo>  explain select 8 = null;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     8     0                    0   Start at 8
1     Integer            8     2     0                    0   r[2]=8
2     Null               0     3     0                    0   r[3]=NULL
3     Integer            1     1     0                    0   r[1]=1
4     Eq                 2     3     6                    0   if r[2]==r[3] goto 6
5     ZeroOrNull         2     1     3                    0   ((r[2]=NULL)|(r[3]=NULL)) ? r[1]=NULL : r[1]=0
6     ResultRow          1     1     0                    0   output=r[1]
7     Halt               0     0     0                    0
8     Transaction        0     0     0                    0
9     Goto               0     1     0                    0
```

Closes #733
2025-01-19 09:09:12 +02:00
Krishna Vishal
870a2ea802 clean up 2025-01-19 06:40:40 +05:30
Krishna Vishal
fa0503f0ce 1. Changes to extension.py
2. chore: cargo fmt
2025-01-19 04:58:05 +05:30
Krishna Vishal
6173aeeb3b 1. Fix merge conflicts
2. change tests for extensions to return error instead of null (Preston)
2025-01-19 04:39:25 +05:30
psvri
e16b3491c4 Fix null compares giving incorrect results 2025-01-19 00:44:38 +05:30
Jorge López
0d6e6a0aa4 syntactic changes: use Result::unwrap_or() instead of manual match 2025-01-18 19:20:11 +01:00
Jorge López
00c503dcf5 syntactic changes: lift return out of ifs 2025-01-18 19:20:11 +01:00
Jorge López
90a14a7bc7 syntactic changes: replace explicit while let Some in iterator with for..in loop (see https://rust-lang.github.io/rust-clippy/master/index.html#while_let_on_iterator) 2025-01-18 19:20:11 +01:00
Jorge López
218a2e6dda syntactic changes: remove unneeded mut, lifetimes,references that get instantaneously dereferenced and casts 2025-01-18 18:43:47 +01:00
Jorge López
2cc8cb9ad8 syntactic changes: use assert_eq!() instead of assert!() for equality comparisons 2025-01-18 18:37:50 +01:00
Jorge López
86a4714711 syntactic changes: remove unneeded paths when the type is already imported 2025-01-18 18:29:12 +01:00
PThorpe92
fc82461eff Complete percentile extension, enable col+delimeter args 2025-01-17 21:15:09 -05:00
PThorpe92
0c737d88f7 Support aggregate functions in Extensions 2025-01-17 14:13:57 -05:00
Pekka Enberg
20ea8fb941 Make Clippy happy 2025-01-16 15:57:08 +02:00
Levy A.
9b8722f38e refactor: more well rounded implementation
`?0` parameters are now handled by the parser.
2025-01-15 16:53:26 -03:00
Levy A.
5de2694834 feat: more parameter support
add `Statement::{parameter_index, parameter_name, parameter_count,
bind_at}`. some refactoring is still needed, this is quite a rough
iteration
2025-01-15 16:51:04 -03:00
Levy A.
d3582a382f fix: small bugs 2025-01-15 16:51:04 -03:00
Levy A.
08c8c655e9 feat: initial implementation of Statement::bind 2025-01-15 16:51:04 -03:00
psvri
845de125db Align MustBeInt logic with sqlite 2025-01-16 00:09:45 +05:30
psvri
d3f28c51f4 Implement ShiftRight 2025-01-15 21:21:51 +05:30
psvri
5b4d82abbf Implement ShiftLeft 2025-01-15 18:54:07 +05:30
PThorpe92
23d9d09b70 Add load_extension function, resolve shared lib extensions 2025-01-14 12:01:07 -05:00
PThorpe92
98eff6cf7a Enable passing arguments to external functions 2025-01-14 07:20:50 -05:00
PThorpe92
3412a3d4c2 Rough design for extension api/draft extension 2025-01-14 07:20:48 -05:00
PThorpe92
0a10d893d9 Sketch out runtime extension loading 2025-01-14 07:18:07 -05:00
Jussi Saurio
d3c2868fcb Merge 'refactor: json functions vdbe code should be consistent with scalar functions' from Jorge Hermo
While working to include new json functions (related to #127) I noticed
that the program step code for json functions https://github.com/tursoda
tabase/limbo/blob/0dceb02ec04241b3772332853bcbfb9eb559adb9/core/vdbe/mod
.rs#L1346 was a bit different from scalar functions's code, where the
match to the inner function is nested https://github.com/tursodatabase/l
imbo/blob/0dceb02ec04241b3772332853bcbfb9eb559adb9/core/vdbe/mod.rs#L142
4
I added the same nesting to the json functions so it is more consistent

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #663
2025-01-14 10:30:33 +02:00
PThorpe92
abb3fda19f Fix all the flaky datetime tests 2025-01-13 19:51:34 -05:00
Jorge Hermo
df3123e128 Merge with main 2025-01-13 22:20:17 +01:00
Pekka Enberg
44550ab749 cargo fmt 2025-01-13 18:58:40 +02:00
Pekka Enberg
c7ea2393b3 Merge 'Store cursor types in program state and remove trait Cursor' from Jussi Saurio
I was planning on starting work on index insertion, but realized we need
to know whether our cursor refers to a table or an index etc., so it
resulted in this refactoring work.
- `cursor_ref` now contains what _type_ of cursor it is (table, index,
pseudo, sorter)
- `program.cursors` is now `program.btree_table_cursors`,
`program.btree_index_cursors` etc and they are unboxed because dynamic
dispatch is no longer necessary
- Cursor trait removed -- 95% of the shit was btree specific anyway, so
I just moved them to `BTreeCursor`. In certain instructions in the VDBE
we expect a btree cursor and in others we expect a pseudo/sorter etc,
lets make that explicit.
- I also removed `BTreeCursor::get_new_rowid()` specific tests that
required macros to generate a mock implementation of the `Cursor` trait
-- main reason is I couldn't figure out how to reimplement this without
the trait, and the second reason is that I don't think we really need
those tests, AND the proc macro is constantly failing in my editor as
well and screwing up `rust-analyzer`

Closes #655
2025-01-13 18:40:46 +02:00
Pekka Enberg
8769bf82a5 Merge 'fix: flaky test in datetime because compare pre-init now.' from Sonny
Removed flaky test that happens sometimes when the CI runner is a bit
slow e.g. in https://github.com/tursodatabase/limbo/actions/runs/1270626
5753/job/35418879041?pr=643
'now' is covered in other test so it should be fine. it is a problem in
time test, for date not so much.
for example it is covered in:
https://github.com/tursodatabase/limbo/blob/bcc85c37a0febdf5c3f29127029f
e10c9906867d/core/vdbe/datetime.rs#L1434C5-L1446C6

Closes #652
2025-01-13 18:39:54 +02:00
Pekka Enberg
1e94dbffcc Merge branch 'main' into json-error-position 2025-01-13 18:21:37 +02:00
Jorge Hermo
016d9d17ab refactor: json functions vdbe 2025-01-13 00:05:14 +01:00
Jussi Saurio
92888c54e9 remove get_new_rowid() specific tests that require mocking 2025-01-11 17:41:10 +02:00
Jussi Saurio
bf48c0ae72 Remove trait Cursor 2025-01-11 17:19:25 +02:00
Jussi Saurio
9909539b9d Store cursor type (table,index,pseudo,sorter) when allocating cursor 2025-01-11 17:04:16 +02:00
sonhmai
46eba6cc77 fix: flaky test in datetime because compare pre-init now.
'now' is covered in other test so it should be fine.
2025-01-11 16:15:58 +07:00
Peter Sooley
b5fed15997 implement json_error_position 2025-01-10 11:12:30 -08:00
Kacper Madej
536fbe9d9e Fix imports 2025-01-10 12:24:08 +07:00
Kacper Madej
743a8b2d94 Merge branch 'main' into right-arrow-json 2025-01-10 11:28:13 +07:00
Kacper Madej
dd533414ef Implement -> and ->> operators for json 2025-01-09 15:38:32 +07:00
Kacper Madej
eebf9bfaac Implement json_type 2025-01-09 11:29:17 +07:00
PThorpe92
345107ce21 Fix precision issue in datetime tests 2025-01-07 09:57:19 -05:00
Jussi Saurio
731ff1480f Simplify working with labels
TLDR: no need to call either of:

program.emit_insn_with_label_dependency() -> just call program.emit_insn()
program.defer_label_resolution() -> just call program.resolve_label()

Changes:

- make BranchOffset an explicit enum (Label, Offset, Placeholder)
- remove program.emit_insn_with_label_dependency() - label dependency is automatically detected
- for label to offset mapping, use a hashmap from label(negative i32) to offset (positive u32)
- resolve all labels in program.build()
- remove program.defer_label_resolution() - all labels are resolved in build()
2025-01-07 12:53:10 +02:00
Pekka Enberg
ba28999d05 Merge 'Add partial support for datetime() function' from Preston Thorpe
This PR adds the `datetime` function, with all the support currently
that date/time have for modifiers, and `julianday` function, as well as
some additional modifiers for date/time/datetime.
There are a couple considerations here, I left a couple comments but
essentially there is going to have to be some more work done to track
the state of the expression during the application of modifiers, to
handle a bunch of edge-cases like re-applying the same timezone modifier
to itself, or converting an integer automatically assumed to be
julianday, into epoch, or `ceiling`/`floor` which will determine
relative addition of time in cases like
```
2024-01-31 +1 month = 2024-03-02
```
which was painful enough to get working to begin with.
I couldn't get the `julianday_converter` library to get the exact same
float precision as sqlite, so function is included that matches their
output, for some reason floating point math + `.floor()` would give the
correct result. They seem to 'round' to 8 decimal places, and I was able
to get this to work with the same output as sqlite, except in cases like
`2234.5`, in which case we return `2234.5000000` because of the `fmt`
precision:
```rust
pub fn exec_julianday(time_value: &OwnedValue) -> Result<String> {
    let dt = parse_naive_date_time(time_value);
    match dt {
        // if we did something heinous like: parse::<f64>().unwrap().to_string()
        // that would solve the precision issue, but dear lord...
        Some(dt) => Ok(format!("{:.1$}", to_julian_day_exact(&dt), 8)),
        None => Ok(String::new()),
    }
}
```
Suggestions would be appreciated on the float precision issue.

Reviewed-by: Sonny <14060682+sonhmai@users.noreply.github.com>

Closes #600
2025-01-05 20:13:13 +02:00
Pekka Enberg
5c52c8b1e9 Merge 'Fix integer overflow output to be same as sqlite3' from Vrishabh
In sqlite3, before arithmetic operation is done, it first checks if the
operation dosent overflow and then does the operation. In case it
overflows it would covert the arguments into floats and then does the
operation as [per code](https://github.com/sqlite/sqlite/blob/ded37f337b
7b2e916657a83732aaec40eb146282/src/vdbe.c#L1875)  . I have done the same
behaviour for limbo.

Closes #612
2025-01-04 17:46:48 +02:00
PThorpe92
ca428b3dda Julianday function and additional tests/comments 2025-01-04 10:42:34 -05:00
PThorpe92
7c4a780cc2 Add DateTime func and support more modifiers 2025-01-04 08:41:03 -05:00