Commit Graph

557 Commits

Author SHA1 Message Date
Pekka Enberg
44e5402464 Merge branch 'main' into feature/noop 2025-01-31 18:49:39 +02:00
Pekka Enberg
3a4cb34606 Merge 'Fix memory leaks, make extension types more efficient' from Preston Thorpe
I was baffled previously, because any time that `free` was called on a
type from an extension, it would hang even when I knew it wasn't in use
any longer, and hadn't been double free'd.
After #737 was merged, I tried it again and noticed that it would no
longer hang... but only for extensions that were staticly linked.
Then I realized that we are using a global allocator, that likely wasn't
getting used in the shared library that is built separately that won't
inherit from our global allocator in core, causing some symbol mismatch
and the subsequent hanging on calls to `free`.
This PR adds the global allocator to extensions behind a feature flag in
the macro that will prevent it from being used in `wasm` and staticly
linked environments where it would conflict with limbos normal global
allocator. This allows us to properly free the memory from returning
extension functions over FFI.
This PR also changes the Extension type to a union field so we can store
int + float values inline without boxing them.
any additional tips or thoughts anyone else has on improving this would
be appreciated 👍

Closes #803
2025-01-30 13:31:17 +02:00
Pekka Enberg
c779537f2f Merge 'Strftime compatibility solved' from Pedro Muniz
This PR closes #787. Chrono offers to format the string from an iterator
of Format Items. I created a custom iterator that only allows formatters
specified by sqlite. This approach however does not address the
inefficient way that julianday is calculated. Also, with this
implementation we avoid having to maintain a separate vendored package
for strftime that may become incompatible with Chrono in the future.

Closes #792
2025-01-30 13:30:11 +02:00
Pekka Enberg
e66648beb8 Merge 'Add support for offset in select queries' from Ben Li
#739
Started adding support for `LIMIT...OFFSET...`
- New `OffsetLimit` opcode
- `OFFSET` is now supported for:
    - `SELECT...LIMIT...OFFSET`
    - `SELECT...GROUP BY...LIMIT...OFFSET`
    - `SELECT...ORDER BY...LIMIT...OFFSET`
    - Subqueries for `SELECT` statements
**In progress/todo**
- [x] Testing
- [x] Handle negative offset value
- **(will make in separate PR)** Add support for
`DELETE...LIMIT...OFFSET`
- **(will make in separate PR)** Use `limit + offset` sum register from
`OffsetLimit` to constrain number of records inserted into sorter

Closes #779
2025-01-30 13:29:49 +02:00
Pekka Enberg
a2ac3132c4 Merge 'Fix SELECT ABS(-9223372036854775808) causes limbo to panic. ' from Krishna Vishal
Now we return `RuntimeError`.  Matches SQLite behavior.
SQLite:
```sql
sqlite> SELECT ABS(-9223372036854775808);
Runtime error: integer overflow
```
Limbo after this fix:
```sql
limbo> SELECT ABS(-9223372036854775808);
Runtime error: integer overflow
```
Closes https://github.com/tursodatabase/limbo/issues/815

Closes #818
2025-01-30 13:25:39 +02:00
Ihor Andrianov
8a01b842a5 fix function import 2025-01-30 04:05:05 +02:00
Ihor Andrianov
d57d9bef6f add function definition 2025-01-29 22:37:04 +02:00
Pekka Enberg
06edf33878 Merge 'json_patch() function implementation' from Ihor Andrianov
First review #820
The function follows RFC 7386 JSON Merge Patch semantics:
* If the patch is null, the target is replaced with null
* If the patch contains a scalar value, the target is replaced with that
value
* If both target and patch are objects, the patch is recursively applied
* null values in the patch result in property removal from the target

Closes #821
2025-01-29 19:54:12 +02:00
krishvishal
f8c1828ddf Matched on i64::checked_abs and changed RuntimeError to IntegerOverflow 2025-01-29 22:34:19 +05:30
Ihor Andrianov
98be735f5a add json_patch to expr and vm 2025-01-29 18:05:39 +02:00
krishvishal
84d47eb582 Fix SELECT ABS(-9223372036854775808) causes limbo to panic. now returns RuntimeError. Fixes https://github.com/tursodatabase/limbo/issues/815 2025-01-29 18:13:58 +05:30
Glauber Costa
8f24d18ad8 implement sqlite_source_id function 2025-01-28 14:55:38 -05:00
PThorpe92
793cdf8bad Fix memory issues, make extension types more efficient 2025-01-27 22:30:31 -05:00
pedrocarlo
515e281480 implement noop 2025-01-26 20:49:23 -03:00
ben594
a3c0602452 Implemented OffsetLimit opcode 2025-01-26 16:40:30 -05:00
pedrocarlo
a94f2bcf9c cargo fmt 2025-01-26 17:10:04 -03:00
pedrocarlo
58cf453d2a strftime compatibility issue solved 2025-01-26 17:01:08 -03:00
Pekka Enberg
089e4e66eb Merge 'Implemented json_valid function' from Harin
Closes #789
2025-01-26 20:48:16 +02:00
Harin
0903b9b019 Implemented JSON valid function 2025-01-26 23:35:47 +05:30
Pekka Enberg
983875c443 core: Remove database header from BTreeCursor
It's already in the pager so use it from there to reduce the size of the
`BTreeCursor` struct.
2025-01-26 16:48:12 +02:00
Pekka Enberg
7967cc5efc core: Kill Rows wrapper struct
It's just an useless wrapper, kill it.
2025-01-26 16:27:19 +02:00
Jussi Saurio
faa6d0c69d use vec for cursors, not map 2025-01-26 14:04:05 +02:00
Jussi Saurio
cdafc9033e ProgramState::registers - no resize 2025-01-26 13:56:09 +02:00
Jussi Saurio
b687cf66eb use bitfield for ended_coroutine 2025-01-26 13:12:47 +02:00
Jussi Saurio
9e32ce6c77 Add Cursor enum and store a single BTreeMap of cursors in ProgramState 2025-01-26 12:57:15 +02:00
Pekka Enberg
8942c38bda core: Fix Statement::reset()
The first rule of writing fast programs: don't use dynamic memory
allocation!

Brings back some performance for the `SELECT 1` micro-benchmark,
although we're still not where we need to be.
2025-01-26 11:47:28 +02:00
Pedro Muniz
9d858f5cb3 Merge branch 'tursodatabase:main' into feature/strftime 2025-01-25 16:23:32 -03:00
pedrocarlo
a316ab51ac feature: implement strftime function 2025-01-25 16:22:53 -03:00
Pekka Enberg
aded7d3896 Merge 'Implement Or and And bytecodes' from Diego Reis
I think it is mostly correct, not so sure how to handle `BLOB`. One
thing that caught my attention is that sqlite seems to have a
optimization for trivial cases, saving some bytecodes, for instance:
![image](https://github.com/user-
attachments/assets/78b78a0d-5ab6-4a9e-aeac-fa97f1fc5c25)
I'm looking that right now.

Closes #777
2025-01-25 08:42:03 +02:00
Diego Reis
e7d95399e3 Add Or bytecode
Take the logical OR of the values in register P1 and P2 and store the answer in register P3. If either P1 or P2 is nonzero (true) then the result is 1 (true) even if the other input is NULL. A NULL and false or two NULLs give a NULL output.
2025-01-25 02:54:14 -03:00
Diego Reis
aff454b5f6 Implement And bytecode
Take the logical AND of the values in registers P1 and P2 and write the result into register P3. If either P1 or P2 is 0 (false) then the result is 0 even if the other input is NULL. A NULL and true or two NULLs give a NULL output.
2025-01-25 02:12:50 -03:00
PThorpe92
545990f806 Support returning column names from prepared statement 2025-01-23 11:02:31 -05:00
Harin
ab3a15e489 Code refactor 2025-01-21 10:02:19 +05:30
Harin
da53cc3821 Added Concat Opcode 2025-01-21 00:29:23 +05:30
Jussi Saurio
2cd9118be6 Fix jump_if_true to be a bool literal in places where it was used as a register number 2025-01-20 17:13:34 +02:00
Jussi Saurio
f88a4d6ac6 Add jump_if_null to cmp insns to account for either operand being NULL 2025-01-20 16:54:39 +02:00
Pekka Enberg
9369f06699 Merge 'Initial support for wal_checkpoint pragma' from Sonny
Wire pragma wal_checkpoint to checkpoint infra
- add basic support for parsing and instruction emitting `pragma
wal_checkpoint;`
- checkpoint opcode for instruction
- checkpoint execution in `virtual machine`
- cli test
Part of #696.
Before
```
limbo> pragma wal_checkpoint;

  × Parse error: Not a valid pragma name
```
After
```
Enter ".help" for usage hints.
limbo> pragma wal_checkpoint;
0|0|0
```
```

Closes #694
2025-01-20 09:57:58 +02:00
Pekka Enberg
bda1e4e6ab Merge 'Add support for json_object function' from Jorge Hermo
Relates to #127.  This PR is still in draft and I have a few left things
to do (tests, improve implementation), just opening it so anyone can
track this work meanwhile.

Closes #664
2025-01-20 09:36:56 +02:00
Pekka Enberg
c25d9a1824 Merge 'Implement Not' from Vrishabh
This PR adds support for Not operator and Opcode.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #748
2025-01-20 09:17:45 +02:00
ben594
6c4ee1e905 Update changes on delete
Remove unwrap
2025-01-19 20:51:16 -05:00
ben594
28ce68091f Modified changes and total_changes scalarfuncs to be more like sqlite
Fmt and clippy

Remove print
2025-01-19 20:51:13 -05:00
sonhmai
e45a807f0e core: allocate 2 registers for checkpoint opcode execution 2025-01-20 08:34:13 +07:00
sonhmai
cb631dafdc feat: wire checkpoint to bytecode execution 2025-01-20 08:34:13 +07:00
sonhmai
66d6291f32 add scaffolding for supporting wal checkpoint 2025-01-20 08:34:13 +07:00
psvri
e616bd5361 Implement Not 2025-01-20 00:21:23 +05:30
Pekka Enberg
f5e5428d45 Merge 'Syntactic improvements' from Jorge López Tello
This is a purely syntactic PR. It doesn't change behavior, just rewrites
some loops and removes unneeded parts, like lifetime annotations and
references. Mainly because the Clippy and IDE warnings get annoying.
Don't worry about the number of commits, I just separated based on type
of change.

Closes #732
2025-01-19 12:17:28 +02:00
Pekka Enberg
3e28541b53 Merge 'Fix null compare operations not giving null' from Vrishabh
In limbo when we do any compare operations like `Eq, gt, lt, gte, lte`
with nulls , we were actually giving the result as true where as sqlite3
gives null. This is because if we had a null, we were incorrectly going
to conditional branch and not increment program by 1. Also the sqlite
generates `ZeroOrNull` op in these cases
(https://github.com/sqlite/sqlite/blob/version-3.45.3/src/expr.c#L4644)
but we were generating a Integer instruction. The below outputs can give
a clearer picture.
This PR aims to fix this.
sqlite3 output
```
SQLite version 3.48.0
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite> select 8 = null;

sqlite> select 8 > null;

sqlite> explain select 8 > null;
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     6     0                    0
1     Integer        1     1     0                    0
2     Gt             3     4     2                    64
3     ZeroOrNull     2     1     3                    0
4     ResultRow      1     1     0                    0
5     Halt           0     0     0                    0
6     Integer        8     2     0                    0
7     Null           0     3     0                    0
8     Goto           0     1     0                    0
sqlite> explain select 8 = null;
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     6     0                    0
1     Integer        1     1     0                    0
2     Eq             3     4     2                    64
3     ZeroOrNull     2     1     3                    0
4     ResultRow      1     1     0                    0
5     Halt           0     0     0                    0
6     Integer        8     2     0                    0
7     Null           0     3     0                    0
8     Goto           0     1     0                    0
```
Limbo Output
```
Limbo v0.0.12
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database
limbo> select 8 = null;
1
limbo> select 8 > null;
1
limbo> explain select 8 > null;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     8     0                    0   Start at 8
1     Integer            8     2     0                    0   r[2]=8
2     Null               0     3     0                    0   r[3]=NULL
3     Integer            1     1     0                    0   r[1]=1
4     Gt                 2     3     6                    0   if r[2]>r[3] goto 6
5     Integer            0     1     0                    0   r[1]=0
6     ResultRow          1     1     0                    0   output=r[1]
7     Halt               0     0     0                    0
8     Transaction        0     0     0                    0
9     Goto               0     1     0                    0
limbo> explain select 8 = null;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     8     0                    0   Start at 8
1     Integer            8     2     0                    0   r[2]=8
2     Null               0     3     0                    0   r[3]=NULL
3     Integer            1     1     0                    0   r[1]=1
4     Eq                 2     3     6                    0   if r[2]==r[3] goto 6
5     Integer            0     1     0                    0   r[1]=0
6     ResultRow          1     1     0                    0   output=r[1]
7     Halt               0     0     0                    0
8     Transaction        0     0     0                    0
9     Goto               0     1     0                    0
limbo>
```
Limbo Output with this PR
```
Limbo v0.0.12
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database
limbo> select 8 = null;

limbo> select 8 > null;

limbo> explain select 8 > null;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     8     0                    0   Start at 8
1     Integer            8     2     0                    0   r[2]=8
2     Null               0     3     0                    0   r[3]=NULL
3     Integer            1     1     0                    0   r[1]=1
4     Gt                 2     3     6                    0   if r[2]>r[3] goto 6
5     ZeroOrNull         2     1     3                    0   ((r[2]=NULL)|(r[3]=NULL)) ? r[1]=NULL : r[1]=0
6     ResultRow          1     1     0                    0   output=r[1]
7     Halt               0     0     0                    0
8     Transaction        0     0     0                    0
9     Goto               0     1     0                    0
limbo>  explain select 8 = null;
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     8     0                    0   Start at 8
1     Integer            8     2     0                    0   r[2]=8
2     Null               0     3     0                    0   r[3]=NULL
3     Integer            1     1     0                    0   r[1]=1
4     Eq                 2     3     6                    0   if r[2]==r[3] goto 6
5     ZeroOrNull         2     1     3                    0   ((r[2]=NULL)|(r[3]=NULL)) ? r[1]=NULL : r[1]=0
6     ResultRow          1     1     0                    0   output=r[1]
7     Halt               0     0     0                    0
8     Transaction        0     0     0                    0
9     Goto               0     1     0                    0
```

Closes #733
2025-01-19 09:09:12 +02:00
Krishna Vishal
870a2ea802 clean up 2025-01-19 06:40:40 +05:30
Krishna Vishal
fa0503f0ce 1. Changes to extension.py
2. chore: cargo fmt
2025-01-19 04:58:05 +05:30
Krishna Vishal
6173aeeb3b 1. Fix merge conflicts
2. change tests for extensions to return error instead of null (Preston)
2025-01-19 04:39:25 +05:30