Commit Graph

351 Commits

Author SHA1 Message Date
Pekka Enberg
9458d1ed14 Merge 'Fix shr instruction' from Nikita Sivukhin
This PR fixes implementation of binary shift right/left instructions.
Before there were a minor incompatibility between limbo and sqlite
implementation in case when right shift second argument were more than
64 and first argument were negative. As sqlite implementation of right
binary shift is sign-extended - so `-1` should be returned in such case
when limbo returned zero.
This PR fixes this bug and also introduce a fuzz tests for arithemtic
expressions. This fuzz test were written with a help of
`GrammarGenerator` which allows to easily define probabilistic context-
free grammar and then later sample random strings from it.

Closes #867
2025-02-03 09:22:39 +02:00
Nikita Sivukhin
2b9220992d fix attempt to add with overflow crash in case of rowid auto-generation 2025-02-02 20:10:58 +04:00
Nikita Sivukhin
43c9fc3c5c fix binary shift implementation 2025-02-02 19:24:22 +04:00
Glauber Costa
a3387cfd5f implement the pragma page_count
To do that, we also have to implement the vdbe opcode Pagecount.
2025-02-01 19:39:46 -05:00
Pekka Enberg
20d3399c71 Merge 'implement is and is not where constraints' from Glauber Costa
The main difference between = and != is how null values are handled.
SQLite passes a flag "NULLEQ" to Eq and Ne to disambiguate that.
In the presence of that flag, NULL = NULL.
Some prep work is done to make sure we can pass a flag instead of a
boolean to Eq and Ne. I looked into the bitflags crate but got a bit
scared with the list of dependencies.
Warning:
The following query produces a different result for Limbo:
```
select * from demo where value is null or id == 2;
```
I strongly suspect the issue is with the OR implementation, though. The
bytecode generated is quite different.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #847
2025-02-01 17:24:11 +02:00
Glauber Costa
c04260ab54 rename Flags to a less ambiguous name
Those Flags in SQLite are global, but it doesn't mean it has to be
the case for us as well.
2025-02-01 08:09:06 -05:00
Pekka Enberg
db29f43d5c Merge 'Simplify bytecode emitters' from Glauber Costa
Instead of always having the caller specify all instructions, this
    work introduces convenience functions into the program builder,
    making the code a lot cleaner.
    Draft for now, as this is done on top of #841

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #844
2025-02-01 09:24:11 +02:00
Glauber Costa
96987db6ca implement is and is not where constraints
The main difference between = and != is how null values are handled.
SQLite passes a flag "NULLEQ" to Eq and Ne to disambiguate that.
In the presence of that flag, NULL = NULL.

Some prep work is done to make sure we can pass a flag instead of a
boolean to Eq and Ne. I looked into the bitflags crate but got a bit
scared with the list of dependencies.
2025-01-31 23:01:49 -05:00
Glauber Costa
f300d2c8e8 rename register for IsNull opcode
Now it has the same name as NotNull, so it is easier to write macros
2025-01-31 19:09:01 -05:00
Pekka Enberg
44e5402464 Merge branch 'main' into feature/noop 2025-01-31 18:49:39 +02:00
Glauber Costa
7aa3cc26ad simplify the writing of bytecode programs
Instead of always having the caller specify all instructions, this
work introduces convenience functions into the program builder,
making the code a lot cleaner.
2025-01-31 11:35:51 -05:00
Pekka Enberg
3a4cb34606 Merge 'Fix memory leaks, make extension types more efficient' from Preston Thorpe
I was baffled previously, because any time that `free` was called on a
type from an extension, it would hang even when I knew it wasn't in use
any longer, and hadn't been double free'd.
After #737 was merged, I tried it again and noticed that it would no
longer hang... but only for extensions that were staticly linked.
Then I realized that we are using a global allocator, that likely wasn't
getting used in the shared library that is built separately that won't
inherit from our global allocator in core, causing some symbol mismatch
and the subsequent hanging on calls to `free`.
This PR adds the global allocator to extensions behind a feature flag in
the macro that will prevent it from being used in `wasm` and staticly
linked environments where it would conflict with limbos normal global
allocator. This allows us to properly free the memory from returning
extension functions over FFI.
This PR also changes the Extension type to a union field so we can store
int + float values inline without boxing them.
any additional tips or thoughts anyone else has on improving this would
be appreciated 👍

Closes #803
2025-01-30 13:31:17 +02:00
Pekka Enberg
c779537f2f Merge 'Strftime compatibility solved' from Pedro Muniz
This PR closes #787. Chrono offers to format the string from an iterator
of Format Items. I created a custom iterator that only allows formatters
specified by sqlite. This approach however does not address the
inefficient way that julianday is calculated. Also, with this
implementation we avoid having to maintain a separate vendored package
for strftime that may become incompatible with Chrono in the future.

Closes #792
2025-01-30 13:30:11 +02:00
Pekka Enberg
e66648beb8 Merge 'Add support for offset in select queries' from Ben Li
#739
Started adding support for `LIMIT...OFFSET...`
- New `OffsetLimit` opcode
- `OFFSET` is now supported for:
    - `SELECT...LIMIT...OFFSET`
    - `SELECT...GROUP BY...LIMIT...OFFSET`
    - `SELECT...ORDER BY...LIMIT...OFFSET`
    - Subqueries for `SELECT` statements
**In progress/todo**
- [x] Testing
- [x] Handle negative offset value
- **(will make in separate PR)** Add support for
`DELETE...LIMIT...OFFSET`
- **(will make in separate PR)** Use `limit + offset` sum register from
`OffsetLimit` to constrain number of records inserted into sorter

Closes #779
2025-01-30 13:29:49 +02:00
Pekka Enberg
a2ac3132c4 Merge 'Fix SELECT ABS(-9223372036854775808) causes limbo to panic. ' from Krishna Vishal
Now we return `RuntimeError`.  Matches SQLite behavior.
SQLite:
```sql
sqlite> SELECT ABS(-9223372036854775808);
Runtime error: integer overflow
```
Limbo after this fix:
```sql
limbo> SELECT ABS(-9223372036854775808);
Runtime error: integer overflow
```
Closes https://github.com/tursodatabase/limbo/issues/815

Closes #818
2025-01-30 13:25:39 +02:00
Ihor Andrianov
8a01b842a5 fix function import 2025-01-30 04:05:05 +02:00
Ihor Andrianov
d57d9bef6f add function definition 2025-01-29 22:37:04 +02:00
Pekka Enberg
06edf33878 Merge 'json_patch() function implementation' from Ihor Andrianov
First review #820
The function follows RFC 7386 JSON Merge Patch semantics:
* If the patch is null, the target is replaced with null
* If the patch contains a scalar value, the target is replaced with that
value
* If both target and patch are objects, the patch is recursively applied
* null values in the patch result in property removal from the target

Closes #821
2025-01-29 19:54:12 +02:00
krishvishal
f8c1828ddf Matched on i64::checked_abs and changed RuntimeError to IntegerOverflow 2025-01-29 22:34:19 +05:30
Ihor Andrianov
98be735f5a add json_patch to expr and vm 2025-01-29 18:05:39 +02:00
krishvishal
84d47eb582 Fix SELECT ABS(-9223372036854775808) causes limbo to panic. now returns RuntimeError. Fixes https://github.com/tursodatabase/limbo/issues/815 2025-01-29 18:13:58 +05:30
Glauber Costa
8f24d18ad8 implement sqlite_source_id function 2025-01-28 14:55:38 -05:00
PThorpe92
793cdf8bad Fix memory issues, make extension types more efficient 2025-01-27 22:30:31 -05:00
pedrocarlo
97c716524b cargo fmt and clippy 2025-01-26 20:51:14 -03:00
pedrocarlo
515e281480 implement noop 2025-01-26 20:49:23 -03:00
ben594
a3c0602452 Implemented OffsetLimit opcode 2025-01-26 16:40:30 -05:00
pedrocarlo
a94f2bcf9c cargo fmt 2025-01-26 17:10:04 -03:00
pedrocarlo
58cf453d2a strftime compatibility issue solved 2025-01-26 17:01:08 -03:00
Pekka Enberg
089e4e66eb Merge 'Implemented json_valid function' from Harin
Closes #789
2025-01-26 20:48:16 +02:00
Harin
0903b9b019 Implemented JSON valid function 2025-01-26 23:35:47 +05:30
Pekka Enberg
983875c443 core: Remove database header from BTreeCursor
It's already in the pager so use it from there to reduce the size of the
`BTreeCursor` struct.
2025-01-26 16:48:12 +02:00
Pekka Enberg
7967cc5efc core: Kill Rows wrapper struct
It's just an useless wrapper, kill it.
2025-01-26 16:27:19 +02:00
Jussi Saurio
faa6d0c69d use vec for cursors, not map 2025-01-26 14:04:05 +02:00
Jussi Saurio
cdafc9033e ProgramState::registers - no resize 2025-01-26 13:56:09 +02:00
Jussi Saurio
b687cf66eb use bitfield for ended_coroutine 2025-01-26 13:12:47 +02:00
Jussi Saurio
9e32ce6c77 Add Cursor enum and store a single BTreeMap of cursors in ProgramState 2025-01-26 12:57:15 +02:00
Pekka Enberg
8942c38bda core: Fix Statement::reset()
The first rule of writing fast programs: don't use dynamic memory
allocation!

Brings back some performance for the `SELECT 1` micro-benchmark,
although we're still not where we need to be.
2025-01-26 11:47:28 +02:00
Pedro Muniz
9d858f5cb3 Merge branch 'tursodatabase:main' into feature/strftime 2025-01-25 16:23:32 -03:00
pedrocarlo
a316ab51ac feature: implement strftime function 2025-01-25 16:22:53 -03:00
Pekka Enberg
aded7d3896 Merge 'Implement Or and And bytecodes' from Diego Reis
I think it is mostly correct, not so sure how to handle `BLOB`. One
thing that caught my attention is that sqlite seems to have a
optimization for trivial cases, saving some bytecodes, for instance:
![image](https://github.com/user-
attachments/assets/78b78a0d-5ab6-4a9e-aeac-fa97f1fc5c25)
I'm looking that right now.

Closes #777
2025-01-25 08:42:03 +02:00
Diego Reis
7902d5f200 Fix Text handling of And bytecode 2025-01-25 03:27:11 -03:00
Diego Reis
e7d95399e3 Add Or bytecode
Take the logical OR of the values in register P1 and P2 and store the answer in register P3. If either P1 or P2 is nonzero (true) then the result is 1 (true) even if the other input is NULL. A NULL and false or two NULLs give a NULL output.
2025-01-25 02:54:14 -03:00
Diego Reis
aff454b5f6 Implement And bytecode
Take the logical AND of the values in registers P1 and P2 and write the result into register P3. If either P1 or P2 is 0 (false) then the result is 0 even if the other input is NULL. A NULL and true or two NULLs give a NULL output.
2025-01-25 02:12:50 -03:00
PThorpe92
545990f806 Support returning column names from prepared statement 2025-01-23 11:02:31 -05:00
Harin
ab3a15e489 Code refactor 2025-01-21 10:02:19 +05:30
Harin
da53cc3821 Added Concat Opcode 2025-01-21 00:29:23 +05:30
Jussi Saurio
2cd9118be6 Fix jump_if_true to be a bool literal in places where it was used as a register number 2025-01-20 17:13:34 +02:00
Jussi Saurio
f88a4d6ac6 Add jump_if_null to cmp insns to account for either operand being NULL 2025-01-20 16:54:39 +02:00
Pekka Enberg
9369f06699 Merge 'Initial support for wal_checkpoint pragma' from Sonny
Wire pragma wal_checkpoint to checkpoint infra
- add basic support for parsing and instruction emitting `pragma
wal_checkpoint;`
- checkpoint opcode for instruction
- checkpoint execution in `virtual machine`
- cli test
Part of #696.
Before
```
limbo> pragma wal_checkpoint;

  × Parse error: Not a valid pragma name
```
After
```
Enter ".help" for usage hints.
limbo> pragma wal_checkpoint;
0|0|0
```
```

Closes #694
2025-01-20 09:57:58 +02:00
Pekka Enberg
bda1e4e6ab Merge 'Add support for json_object function' from Jorge Hermo
Relates to #127.  This PR is still in draft and I have a few left things
to do (tests, improve implementation), just opening it so anyone can
track this work meanwhile.

Closes #664
2025-01-20 09:36:56 +02:00