Commit Graph

386 Commits

Author SHA1 Message Date
Jussi Saurio
fcfee24c50 Remove mark_last_insn_constant() from places where it is not safe to do so 2025-01-14 16:06:49 +02:00
PThorpe92
9c208dc866 Add tests for first extension 2025-01-14 07:27:35 -05:00
PThorpe92
98eff6cf7a Enable passing arguments to external functions 2025-01-14 07:20:50 -05:00
PThorpe92
3412a3d4c2 Rough design for extension api/draft extension 2025-01-14 07:20:48 -05:00
Levy A.
2f2c96fa2c chore: cargo fmt 2025-01-13 21:31:33 -03:00
Levy A.
eff5de50c5 refactor: make translate_* functions accept ProgramBuilder
simplifies function signatures and allows attaching more context to
ProgramStatus on `translate::translate`, useful for query parameters.
2025-01-13 20:41:56 -03:00
Pekka Enberg
c7ea2393b3 Merge 'Store cursor types in program state and remove trait Cursor' from Jussi Saurio
I was planning on starting work on index insertion, but realized we need
to know whether our cursor refers to a table or an index etc., so it
resulted in this refactoring work.
- `cursor_ref` now contains what _type_ of cursor it is (table, index,
pseudo, sorter)
- `program.cursors` is now `program.btree_table_cursors`,
`program.btree_index_cursors` etc and they are unboxed because dynamic
dispatch is no longer necessary
- Cursor trait removed -- 95% of the shit was btree specific anyway, so
I just moved them to `BTreeCursor`. In certain instructions in the VDBE
we expect a btree cursor and in others we expect a pseudo/sorter etc,
lets make that explicit.
- I also removed `BTreeCursor::get_new_rowid()` specific tests that
required macros to generate a mock implementation of the `Cursor` trait
-- main reason is I couldn't figure out how to reimplement this without
the trait, and the second reason is that I don't think we really need
those tests, AND the proc macro is constantly failing in my editor as
well and screwing up `rust-analyzer`

Closes #655
2025-01-13 18:40:46 +02:00
Pekka Enberg
1e94dbffcc Merge branch 'main' into json-error-position 2025-01-13 18:21:37 +02:00
Jussi Saurio
9909539b9d Store cursor type (table,index,pseudo,sorter) when allocating cursor 2025-01-11 17:04:16 +02:00
Jussi Saurio
e782f30ffe Add expect_arguments_min!() and use expect_arguments_* macros more universally 2025-01-11 09:55:07 +02:00
Peter Sooley
b5fed15997 implement json_error_position 2025-01-10 11:12:30 -08:00
Kacper Madej
7efa87ff31 Merge branch 'main' into right-arrow-json 2025-01-10 19:25:56 +07:00
Jussi Saurio
1bcdf99eab core/optimizer: do expression rewriting on all expressions 2025-01-10 10:04:07 +02:00
Kacper Madej
1a46988a73 Fix hardcoded numbers in macros 2025-01-10 14:50:38 +07:00
Kacper Madej
91d4ac3ac0 Fix expression chaining 2025-01-10 12:14:57 +07:00
Kacper Madej
743a8b2d94 Merge branch 'main' into right-arrow-json 2025-01-10 11:28:13 +07:00
Pekka Enberg
317acb842b Merge 'Implement json_type' from Kacper Madej
Closes #631
2025-01-09 14:25:32 +02:00
Kacper Madej
dd533414ef Implement -> and ->> operators for json 2025-01-09 15:38:32 +07:00
Kacper Madej
eebf9bfaac Implement json_type 2025-01-09 11:29:17 +07:00
PThorpe92
97d0fc68b2 Add macro and helper to clean up expression translations 2025-01-08 19:29:56 -05:00
PThorpe92
183797898b Add tests for nested conditional expressions 2025-01-08 17:19:37 -05:00
PThorpe92
fa0e7d5729 Support nested parenthesized conditional exprs in translator 2025-01-08 17:16:17 -05:00
Jussi Saurio
b062a5f528 Fix bug with column being considered rowid alias based on 'primary_key' 2025-01-08 08:20:13 +02:00
Jussi Saurio
925bd62cbc fix logic bug in check_index_scan() that swapped lhs/rhs but not the comparison op 2025-01-08 08:20:13 +02:00
Jussi Saurio
4a58898863 Rename eliminate_between to rewrite_exprs and add true/false->1/0 case there 2025-01-08 08:20:13 +02:00
Jussi Saurio
731ff1480f Simplify working with labels
TLDR: no need to call either of:

program.emit_insn_with_label_dependency() -> just call program.emit_insn()
program.defer_label_resolution() -> just call program.resolve_label()

Changes:

- make BranchOffset an explicit enum (Label, Offset, Placeholder)
- remove program.emit_insn_with_label_dependency() - label dependency is automatically detected
- for label to offset mapping, use a hashmap from label(negative i32) to offset (positive u32)
- resolve all labels in program.build()
- remove program.defer_label_resolution() - all labels are resolved in build()
2025-01-07 12:53:10 +02:00
Pekka Enberg
ba28999d05 Merge 'Add partial support for datetime() function' from Preston Thorpe
This PR adds the `datetime` function, with all the support currently
that date/time have for modifiers, and `julianday` function, as well as
some additional modifiers for date/time/datetime.
There are a couple considerations here, I left a couple comments but
essentially there is going to have to be some more work done to track
the state of the expression during the application of modifiers, to
handle a bunch of edge-cases like re-applying the same timezone modifier
to itself, or converting an integer automatically assumed to be
julianday, into epoch, or `ceiling`/`floor` which will determine
relative addition of time in cases like
```
2024-01-31 +1 month = 2024-03-02
```
which was painful enough to get working to begin with.
I couldn't get the `julianday_converter` library to get the exact same
float precision as sqlite, so function is included that matches their
output, for some reason floating point math + `.floor()` would give the
correct result. They seem to 'round' to 8 decimal places, and I was able
to get this to work with the same output as sqlite, except in cases like
`2234.5`, in which case we return `2234.5000000` because of the `fmt`
precision:
```rust
pub fn exec_julianday(time_value: &OwnedValue) -> Result<String> {
    let dt = parse_naive_date_time(time_value);
    match dt {
        // if we did something heinous like: parse::<f64>().unwrap().to_string()
        // that would solve the precision issue, but dear lord...
        Some(dt) => Ok(format!("{:.1$}", to_julian_day_exact(&dt), 8)),
        None => Ok(String::new()),
    }
}
```
Suggestions would be appreciated on the float precision issue.

Reviewed-by: Sonny <14060682+sonhmai@users.noreply.github.com>

Closes #600
2025-01-05 20:13:13 +02:00
Pekka Enberg
4a5b6b43bd Merge 'Fix quote escape in literals' from Vrishabh
Previously we were not escaping the quotes properly in stirng literals
as shown below. The PR fixes that
limbo output without this PR
```
limbo> select '''a';
''a
limbo>  explain select '''a';
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     4     0                    0   Start at 4
1     String8            0     1     0     ''a            0   r[1]='''a'
2     ResultRow          1     1     0                    0   output=r[1]
3     Halt               0     0     0                    0
4     Transaction        0     0     0                    0
5     Goto               0     1     0                    0
```
sqlite3 output
```
sqlite> select '''a';
'a
sqlite> explain select '''a';
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     4     0                    0
1     String8        0     1     0     'a             0
2     ResultRow      1     1     0                    0
3     Halt           0     0     0                    0
4     Goto           0     1     0                    0
```
limbo output with this PR
```
limbo> select '''a';
'a
limbo> explain select '''a';
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     4     0                    0   Start at 4
1     String8            0     1     0     'a             0   r[1]=''a'
2     ResultRow          1     1     0                    0   output=r[1]
3     Halt               0     0     0                    0
4     Transaction        0     0     0                    0
5     Goto               0     1     0                    0
```

Closes #616
2025-01-05 20:11:48 +02:00
Jussi Saurio
f434b24e63 Fix limbo/core to work with new boxed ast types 2025-01-05 13:51:34 +02:00
psvri
9340c8f0b1 Change sanitize_string comments to doc string 2025-01-05 17:16:18 +05:30
psvri
a11f4b2b10 Refactor escape string literal logic to a function 2025-01-05 14:21:11 +05:30
psvri
2d84956fda Fix quote escape in literals 2025-01-05 01:35:29 +05:30
PThorpe92
ca428b3dda Julianday function and additional tests/comments 2025-01-04 10:42:34 -05:00
PThorpe92
7c4a780cc2 Add DateTime func and support more modifiers 2025-01-04 08:41:03 -05:00
Jussi Saurio
9a8156753e core/translate: break emitter.rs into smaller modules 2025-01-04 14:52:46 +02:00
Jussi Saurio
1b61749c0f feat/core/translate: create automatic index in CREATE TABLE when necessary 2025-01-04 13:54:44 +02:00
Jussi Saurio
e31317fbb5 emitter.rs: use way less arguments to functions 2025-01-04 12:59:30 +02:00
Jussi Saurio
d2b73e8492 emitter.rs: make t_ctx always be the second argument to any functions 2025-01-04 12:32:50 +02:00
Jussi Saurio
d1f74fa3cb Emitter cleanup part 2: add Resolver 2025-01-04 12:23:19 +02:00
Jussi Saurio
23f1858239 translatectx: consistent naming 2025-01-04 10:39:32 +02:00
Jussi Saurio
1a01487872 left join metadata: consistent naming 2025-01-04 10:39:32 +02:00
Jussi Saurio
2f129402e8 sorter data register: consistent naming 2025-01-04 10:39:32 +02:00
Jussi Saurio
0c572cda3c more consistent function naming 2025-01-04 10:39:32 +02:00
Jussi Saurio
9c1d952e3b consistent names for group by regs and labels 2025-01-04 10:39:32 +02:00
Jussi Saurio
28b1754cf4 group_by_emit: add comments to labels and registers 2025-01-04 10:39:32 +02:00
Jussi Saurio
9ccbb69618 remove fields from GroupByMetadata that can be function-local 2025-01-04 10:39:32 +02:00
Pekka Enberg
bd5f081ea8 Merge 'Add support for changes() and total_changes() functions' from Lemon-Peppermint
#525
- Adds the necessary `ScalarFunc` variants to support the `changes()` &
`total_changes()` SQLite function.
- Adds the necessary fields to the `Connection` struct to track changes.
- Modify the `InsertAwait` OpCode behaviour to affect the changes
counter.

Closes #589
2025-01-04 10:14:06 +02:00
Jussi Saurio
a934ead904 Merge 'Json extract' from Kacper Madej
Implements the `json_extract` function.
In the meantime, the json path has already been implemented by
@petersooley in https://github.com/tursodatabase/limbo/pull/555 which is
a requirement for `json_extract`.
However, this PR takes a different approach and parses the JSON path
using the JSON grammar, because there are a lot of quirks in how a JSON
`key` can look (see the JSON grammar in the Pest file).
The downside is that it allocates more memory than the current
implementation, but might be easier to maintain in the long run.
I included a lot of tests with some quirky behavior of the
`json_extract` (some of them still need some work). I also noticed that
these changed between sqlite versions (had `SQLite 3.43.2` locally and
`3.45` gave different results). Due to this, I'm not sure how much value
there is in trying to be fully compatible with SQLite. Perhaps the
approach taken by @petersooley solves 99% of use-cases?

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #524
2025-01-03 23:53:29 +02:00
Lemon-Peppermint
9109dbf8ec Add changes tracking to the 'Connection' struct + 'Insn::InsertAwait' now affects changes counter 2025-01-03 00:04:20 +02:00
Lemon-Peppermint
abd8e6af43 Add early 'changes()' & 'total_changes()' support 2025-01-02 23:35:19 +02:00