Commit Graph

1986 Commits

Author SHA1 Message Date
Pekka Enberg
c8a979eb4b core: Move re-exports at top of lib.rs
Clean up the code a bit by moving re-exports at the top of lib.rs to
make them more visible to the reader.
2025-01-19 11:34:59 +02:00
Pekka Enberg
0561ff1545 Merge 'Fix scalar API in extensions, add documentation and error handling' from Preston Thorpe
Closes #728
Changes the API to one macro/annotation on the relevant function
```rust
#[scalar(name = "uuid4_str", alias = "gen_random_uuid")]
fn uuid4_str(_args: &[Value]) -> Value {
    let uuid = uuid::Uuid::new_v4().to_string();
    Value::from_text(uuid)
}

register_extension! {
    scalars: { uuid4_str, uuid4 }
}
```
The only downside of this, is that for functions that use their
arguments, because this is not a trait, there is not really a way of
enforcing the function signature like there is with the other way.
Documentation has been added for this in the `scalar` macro, so
hopefully will not be an issue.
Also this PR cleans up the Aggregate API by changing the `args` and
`name` functions to constant associated types, as well as adds some
error handling and documentation.
```rust
impl AggFunc for Median {
    type State = Vec<f64>;
    const NAME: &'static str = "median";
    const ARGS: i32 = 1;

    fn step(state: &mut Self::State, args: &[Value]) {
        if let Some(val) = args.first().and_then(Value::to_float) {
            state.push(val);
        }
    }
//.. etc
```

Closes #735
2025-01-19 09:53:31 +02:00
Pekka Enberg
3e28541b53 Merge 'Fix null compare operations not giving null' from Vrishabh
In limbo when we do any compare operations like `Eq, gt, lt, gte, lte`
with nulls , we were actually giving the result as true where as sqlite3
gives null. This is because if we had a null, we were incorrectly going
to conditional branch and not increment program by 1. Also the sqlite
generates `ZeroOrNull` op in these cases
(https://github.com/sqlite/sqlite/blob/version-3.45.3/src/expr.c#L4644)
but we were generating a Integer instruction. The below outputs can give
a clearer picture.
This PR aims to fix this.
sqlite3 output
```
SQLite version 3.48.0
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite> select 8 = null;

sqlite> select 8 > null;

sqlite> explain select 8 > null;
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     6     0                    0
1     Integer        1     1     0                    0
2     Gt             3     4     2                    64
3     ZeroOrNull     2     1     3                    0
4     ResultRow      1     1     0                    0
5     Halt           0     0     0                    0
6     Integer        8     2     0                    0
7     Null           0     3     0                    0
8     Goto           0     1     0                    0
sqlite> explain select 8 = null;
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     6     0                    0
1     Integer        1     1     0                    0
2     Eq             3     4     2                    64
3     ZeroOrNull     2     1     3                    0
4     ResultRow      1     1     0                    0
5     Halt           0     0     0                    0
6     Integer        8     2     0                    0
7     Null           0     3     0                    0
8     Goto           0     1     0                    0
```
Limbo Output
```
Limbo v0.0.12
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database
limbo> select 8 = null;
1
limbo> select 8 > null;
1
limbo> explain select 8 > null;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     8     0                    0   Start at 8
1     Integer            8     2     0                    0   r[2]=8
2     Null               0     3     0                    0   r[3]=NULL
3     Integer            1     1     0                    0   r[1]=1
4     Gt                 2     3     6                    0   if r[2]>r[3] goto 6
5     Integer            0     1     0                    0   r[1]=0
6     ResultRow          1     1     0                    0   output=r[1]
7     Halt               0     0     0                    0
8     Transaction        0     0     0                    0
9     Goto               0     1     0                    0
limbo> explain select 8 = null;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     8     0                    0   Start at 8
1     Integer            8     2     0                    0   r[2]=8
2     Null               0     3     0                    0   r[3]=NULL
3     Integer            1     1     0                    0   r[1]=1
4     Eq                 2     3     6                    0   if r[2]==r[3] goto 6
5     Integer            0     1     0                    0   r[1]=0
6     ResultRow          1     1     0                    0   output=r[1]
7     Halt               0     0     0                    0
8     Transaction        0     0     0                    0
9     Goto               0     1     0                    0
limbo>
```
Limbo Output with this PR
```
Limbo v0.0.12
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database
limbo> select 8 = null;

limbo> select 8 > null;

limbo> explain select 8 > null;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     8     0                    0   Start at 8
1     Integer            8     2     0                    0   r[2]=8
2     Null               0     3     0                    0   r[3]=NULL
3     Integer            1     1     0                    0   r[1]=1
4     Gt                 2     3     6                    0   if r[2]>r[3] goto 6
5     ZeroOrNull         2     1     3                    0   ((r[2]=NULL)|(r[3]=NULL)) ? r[1]=NULL : r[1]=0
6     ResultRow          1     1     0                    0   output=r[1]
7     Halt               0     0     0                    0
8     Transaction        0     0     0                    0
9     Goto               0     1     0                    0
limbo>  explain select 8 = null;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     8     0                    0   Start at 8
1     Integer            8     2     0                    0   r[2]=8
2     Null               0     3     0                    0   r[3]=NULL
3     Integer            1     1     0                    0   r[1]=1
4     Eq                 2     3     6                    0   if r[2]==r[3] goto 6
5     ZeroOrNull         2     1     3                    0   ((r[2]=NULL)|(r[3]=NULL)) ? r[1]=NULL : r[1]=0
6     ResultRow          1     1     0                    0   output=r[1]
7     Halt               0     0     0                    0
8     Transaction        0     0     0                    0
9     Goto               0     1     0                    0
```

Closes #733
2025-01-19 09:09:12 +02:00
Pekka Enberg
cdcc98540a cargo fmt 2025-01-19 08:52:01 +02:00
Pekka Enberg
e2efdf724e Merge 'Fix panics on invalid aggregate function arguments' from Krishna Vishal
This pull request fixes panics when there's an aggregate function call
with too few or too many arguments.

Closes #726
2025-01-19 08:51:55 +02:00
Krishna Vishal
5cf78b7d54 chore: clippy remove unused imports 2025-01-19 07:18:31 +05:30
Krishna Vishal
acad562c07 Remove unnecessary Result 2025-01-19 06:50:00 +05:30
Krishna Vishal
870a2ea802 clean up 2025-01-19 06:40:40 +05:30
Krishna Vishal
fa0503f0ce 1. Changes to extension.py
2. chore: cargo fmt
2025-01-19 04:58:05 +05:30
Krishna Vishal
31856315fe Fix merge conflicts 2025-01-19 04:45:14 +05:30
Krishna Vishal
6173aeeb3b 1. Fix merge conflicts
2. change tests for extensions to return error instead of null (Preston)
2025-01-19 04:39:25 +05:30
Krishna Vishal
ca097b1972 Remove unused import 2025-01-19 04:35:21 +05:30
Krishna Vishal
027803aabf Refactor code 2025-01-19 04:35:21 +05:30
Krishna Vishal
68553904c7 Converted the unconditional unwrap to a match which handles the case when the function is COUNT and args are None and replaces the args. Solves https://github.com/tursodatabase/limbo/issues/725 2025-01-19 04:35:21 +05:30
Krishna Vishal
776aecbce0 1. All aggregate functions now validate their args and return Parse error for wrong number of args.
2. Unknown functions now return `no such function: unknown_function`
2025-01-19 03:00:18 +05:30
PThorpe92
bcd3ae2bd7 Add documentation to core/ext and relevant macros 2025-01-18 16:04:28 -05:00
PThorpe92
9c47379927 Fix deref null ptr in Debug impl on external types 2025-01-18 15:23:33 -05:00
PThorpe92
956320b7d0 Fix scalar API in extensions, add some error handling 2025-01-18 15:19:35 -05:00
Krishna Vishal
ed5c8ddcf0 Remove unused import 2025-01-19 01:02:46 +05:30
Krishna Vishal
4c1f4d71d6 Refactor code 2025-01-19 01:02:41 +05:30
Krishna Vishal
bff3af6326 Converted the unconditional unwrap to a match which handles the case when the function is COUNT and args are None and replaces the args. Solves https://github.com/tursodatabase/limbo/issues/725 2025-01-19 01:02:37 +05:30
psvri
e16b3491c4 Fix null compares giving incorrect results 2025-01-19 00:44:38 +05:30
psvri
554244209d Add Null comparision tests 2025-01-19 00:39:10 +05:30
Pekka Enberg
4a41736f89 Merge 'Implement IsNot operator' from Vrishabh
This PR adds support for IsNot Operator which was not working as shown
below.
```
❯ ./target/debug/limbo
Limbo v0.0.12
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database
limbo> select 1 is not 0;
thread 'main' panicked at core\translate\expr.rs:626:40:
not yet implemented: IsNot
stack backtrace:
```

Closes #731
2025-01-18 19:39:57 +02:00
psvri
b966351e1f Implement IsNot operator 2025-01-18 22:49:09 +05:30
Pekka Enberg
315d225a85 Merge 'Implement Is operator' from Vrishabh
This PR adds support for `Is` Operator which was not working as shown
below. I have also added compare.test for all compare operations on
which we can expand upon.
```
❯ .\target\debug\limbo.exe
Limbo v0.0.12
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database
limbo> select 1 is 2;
thread 'main' panicked at core\translate\expr.rs:613:40:
not yet implemented: Is
```

Closes #729
2025-01-18 19:05:39 +02:00
psvri
5a13f0790f Implement is operator 2025-01-18 15:57:26 +05:30
psvri
bfadd30f54 Add compare tests 2025-01-18 15:51:31 +05:30
Jussi Saurio
a48c9033e2 Merge pull request #724 from PThorpe92/extdocs
Add documentation for extensions/core
2025-01-18 11:08:35 +02:00
Jussi Saurio
af039ffa6e Merge 'Initial support for aggregate functions in extensions' from Preston Thorpe
#708
This PR adds basic support for the following API for defining
Aggregates, and changes the existing API for scalars.
```rust
register_extension! {
    scalars: { Double },
    aggregates: { MedianState },
}

#[derive(ScalarDerive)]
struct Double;

impl Scalar for Double {
    fn name(&self) -> &'static str {
        "double"
    }
    fn call(&self, args: &[Value]) -> Value {
        if let Some(arg) = args.first() {
            match arg.value_type() {
                ValueType::Float => {
                    let val = arg.to_float().unwrap();
                    Value::from_float(val * 2.0)
                }
                ValueType::Integer => {
                    let val = arg.to_integer().unwrap();
                    Value::from_integer(val * 2)
                }
                _ => {
                    println!("arg: {:?}", arg);
                    Value::null()
                }
            }
        } else {
            Value::null()
        }
    }
}

#[derive(AggregateDerive)]
struct MedianState;

impl AggFunc for MedianState {
    type State = Vec<f64>;

    fn name(&self) -> &'static str {
        "median"
    }
    fn args(&self) -> i32 { 1 }

    fn step(state: &mut Self::State, args: &[Value]) {
        if let Some(val) = args.first().and_then(Value::to_float) {
            state.push(val);
        }
    }

    fn finalize(state: Self::State) -> Value {
        if state.is_empty() {
            return Value::null();
        }
        let mut sorted = state;
        sorted.sort_by(|a, b| a.partial_cmp(b).unwrap());
        let len = sorted.len();
        if len % 2 == 1 {
            Value::from_float(sorted[len / 2])
        } else {
            let mid1 = sorted[len / 2 - 1];
            let mid2 = sorted[len / 2];
            Value::from_float((mid1 + mid2) / 2.0)
        }
    }
}

```
I know it's a bit more verbose than the previous version, but I think in
the long run this will be more robust, and it matches the aggregate API
of implementing a trait on a struct that you derive the relevant trait
on.
Also this allows for better registration of functions, I think passing
in the struct identifiers just feels much better than the `"func_name"
=> function_ptr`

Closes #721
2025-01-18 11:07:06 +02:00
Pekka Enberg
7f372d5498 Merge 'bindings/java: Implement minimal execute method ' from Kim Seon Woo
## Purpose of this PR
- Minimal implementation of `execute`
## Changes
### Java side
- Implement `execute`
- Along the way, rename classes and methods which have the same meanings
  - `fileName` -> `filePath`
  - unify file names for rust code to java code e.g.
`limbo_connection.rs` will match `LimboConnection.java`
### Rust side
- Replace `pointer to struct` and `struct to pointer` functions close
together
- Rename rust files to match java files
## Note
- Implementation differs from that of `sqlite-java`. It's because we can
easily add JNI code and thereby implementing bindings is less restriced.
## References
https://github.com/tursodatabase/limbo/issues/615

Closes #723
2025-01-18 09:05:37 +02:00
Pekka Enberg
51e90ba5dd Merge 'Fix SQL comment handling Limbo shell' from Clyde
This PR improves comment handling in Limbo to precisely match SQLite's
behavior:
Fixes some edge cases involving #711
Inline comments mess up queries --
![image](https://github.com/user-
attachments/assets/10a90c39-a9b7-49e4-a018-1489914a5b64)
Query in the left terminal is current limbo state, upper right is limbo
in the state of this PR
and lower right is sqlite behavior.
![image](https://github.com/user-
attachments/assets/2f1b369a-0495-415e-a091-5b6972488f50)
Added support for inline comments using "--" syntax
Comments are now properly stripped before query execution
Maintains correct query execution when comments appear mid-query
Preserves multiline query functionality with comments
Ensures consistent behavior between pasted and typed queries
Testing:
Added test cases for single-line comments
Added test cases for inline comments
Added test cases for multiline queries with comments
Verified behavior matches SQLite CLI
![image](https://github.com/user-
attachments/assets/225772e9-03c4-472d-8a17-0f46e35c34ba)

Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #722
2025-01-18 08:45:57 +02:00
PThorpe92
b57308003e Handle freeing memory in finalize, remove unused free fn 2025-01-17 21:54:25 -05:00
PThorpe92
fc82461eff Complete percentile extension, enable col+delimeter args 2025-01-17 21:15:09 -05:00
PThorpe92
dc16ed1ef6 Add documentation for extensions/core 2025-01-17 20:43:10 -05:00
김선우
ab23e20732 Revert java image. Disable some failing test for now. 2025-01-18 09:52:42 +09:00
김선우
6542fefd83 Change java image 2025-01-18 09:49:02 +09:00
김선우
aa88dd5d1a Print out yest results while build 2025-01-18 09:42:48 +09:00
김선우
eaa8743c36 Nit 2025-01-18 09:16:09 +09:00
김선우
39245f35cc Add TODOs 2025-01-18 09:09:36 +09:00
김선우
5b9a158db1 Remove unused methods 2025-01-18 09:09:36 +09:00
김선우
a3a31e787c Initial pass on step function 2025-01-18 09:09:36 +09:00
김선우
5fc5f650cd Extract set_err_msg_and_throw_exception to utils.rs 2025-01-18 09:09:36 +09:00
김선우
7028d963ba Remove unused methods for now 2025-01-18 09:09:36 +09:00
김선우
f6ec2252cf Group "pointer to struct" and "struct to pointer" functions 2025-01-18 09:09:36 +09:00
김선우
9765eaba52 Implement prepare 2025-01-18 09:09:36 +09:00
김선우
b77bf879f7 Implement prepare on java side 2025-01-18 09:09:36 +09:00
김선우
3e2e998060 Rename fileName to filePath for clarity 2025-01-18 09:09:36 +09:00
김선우
0819963b2f Implement rust side connect and prepare function 2025-01-18 09:09:36 +09:00
김선우
7e78ec448b Clean up error code related classes 2025-01-18 09:09:36 +09:00