Commit Graph

1242 Commits

Author SHA1 Message Date
sonhmai
2d4bf2eb62 core: move pragma statement bytecode generator to its own file. 2025-02-03 09:21:14 +07:00
Pekka Enberg
dbb7d1a6ba Merge 'Pagecount' from Glauber Costa
This PR implements the Pagecount pragma, as well as its associated
bytecode opcode

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #819
2025-02-02 09:32:18 +02:00
Glauber Costa
a3387cfd5f implement the pragma page_count
To do that, we also have to implement the vdbe opcode Pagecount.
2025-02-01 19:39:46 -05:00
Nikita Sivukhin
1bd8b4ef7a pass null_eq flag for instructions generated for expressions (not in the conditions) 2025-02-02 02:51:51 +04:00
Nikita Sivukhin
c7aed22e39 null_eq flag disable effect of jump_if_null flag - so it makes no sense to set them both 2025-02-02 02:29:02 +04:00
Nikita Sivukhin
478ee6be8d remove null optimization which didn't check for jump_if_condition_is_true flag
- limbo already store constants only once and more clever optimizations
  better to do with generic optimizator and not manually
2025-02-02 02:28:07 +04:00
Pekka Enberg
20d3399c71 Merge 'implement is and is not where constraints' from Glauber Costa
The main difference between = and != is how null values are handled.
SQLite passes a flag "NULLEQ" to Eq and Ne to disambiguate that.
In the presence of that flag, NULL = NULL.
Some prep work is done to make sure we can pass a flag instead of a
boolean to Eq and Ne. I looked into the bitflags crate but got a bit
scared with the list of dependencies.
Warning:
The following query produces a different result for Limbo:
```
select * from demo where value is null or id == 2;
```
I strongly suspect the issue is with the OR implementation, though. The
bytecode generated is quite different.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #847
2025-02-01 17:24:11 +02:00
Glauber Costa
c04260ab54 rename Flags to a less ambiguous name
Those Flags in SQLite are global, but it doesn't mean it has to be
the case for us as well.
2025-02-01 08:09:06 -05:00
Pekka Enberg
db29f43d5c Merge 'Simplify bytecode emitters' from Glauber Costa
Instead of always having the caller specify all instructions, this
    work introduces convenience functions into the program builder,
    making the code a lot cleaner.
    Draft for now, as this is done on top of #841

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #844
2025-02-01 09:24:11 +02:00
Glauber Costa
96987db6ca implement is and is not where constraints
The main difference between = and != is how null values are handled.
SQLite passes a flag "NULLEQ" to Eq and Ne to disambiguate that.
In the presence of that flag, NULL = NULL.

Some prep work is done to make sure we can pass a flag instead of a
boolean to Eq and Ne. I looked into the bitflags crate but got a bit
scared with the list of dependencies.
2025-01-31 23:01:49 -05:00
Glauber Costa
f300d2c8e8 rename register for IsNull opcode
Now it has the same name as NotNull, so it is easier to write macros
2025-01-31 19:09:01 -05:00
Pekka Enberg
44e5402464 Merge branch 'main' into feature/noop 2025-01-31 18:49:39 +02:00
Glauber Costa
7aa3cc26ad simplify the writing of bytecode programs
Instead of always having the caller specify all instructions, this
work introduces convenience functions into the program builder,
making the code a lot cleaner.
2025-01-31 11:35:51 -05:00
Glauber Costa
b37317f68b avoid allocations during pragma_list
If we keep the pragma list sorted when declaring it, we can avoid
a vector allocation when printing the pragma_list.
2025-01-31 11:35:51 -05:00
Pekka Enberg
d8a9c57d3a Merge 'Fix table with single column PRIMARY KEY to not create extra btree' from Krishna Vishal
The error is due to comparing the PRIMARY KEY's name to INTEGER when in
it was all in lowercase. This was causing `needs_auto_index` to be set
to `true`.
After the fix:
```
/limbo /tmp/sc2-limbo.db
Limbo v0.0.13
Enter ".help" for usage hints.
limbo> CREATE TABLE temp (t1 integer, primary key (t1));

hexdump -s 28 -n 4 /tmp/sc2-limbo.db
000001c 0000 0200 -- matches SQLite
0000020
```
Closes https://github.com/tursodatabase/limbo/issues/824

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #830
2025-01-31 18:33:28 +02:00
Glauber Costa
a7cc367c1f implement pragma pragma_list
List all available pragmas (Except pragma_list)
2025-01-31 06:44:56 -05:00
Glauber Costa
62efbde661 use strum package to simplify PragmaName enum management
The pragma list will only grow. The strum crate can be used to:
* automatically convert to string from enum
* automatically convert to enum from string
* implement an iterator over all elements of the enum
2025-01-31 06:44:56 -05:00
Pekka Enberg
7f0274e48f Merge 'Table info' from Glauber Costa
This implements the table_info pragma, allowing us to fetch information
about columns present in a table.

Closes #837
2025-01-31 08:46:27 +02:00
Pekka Enberg
053a1acef1 Merge 'Refactor cursor to support multiple state machines' from Alex Miller
This is mostly refactoring Cursor.write_info to instead be an enum,
where one of the options is a WriteInfo.  This permits one to add other
state machines to Cursor, and I added the state needed for Count as an
example, but all the testing for count's implementation depends on
ANALYZE #656 working end-to-end (to some degree) so that one can write a
SQL test for it.
But this code seems conflict-prone, so it seems better to get it in
sooner than later.
I also finally understood what the point of RefCell is from fighting
with rust on this, so that was nice.

Closes #836
2025-01-31 08:45:01 +02:00
Levy A.
5acd7a5ea8 side by side comparison with sqlite 2025-01-31 03:11:07 -03:00
krishvishal
8b2393fcef Check for if a column is in descending order to add an automatic primary key index. 2025-01-31 08:25:54 +05:30
Glauber Costa
016b815b59 implement pragma table_info
Both () and = variants covered. It is important to make sure that
the transaction is a read transaction, so we cannot hide all that logic
inside update_pragma, and have to make our decision before that.
2025-01-30 20:00:20 -05:00
Glauber Costa
249a8cf8d2 keep type information as a string in column metadata
SQLite holds on to it deeply, for example:

sqlite> create table a(a int);
sqlite> create table b(b integer);
sqlite> create table c(c glauber);

sqlite> pragma table_info=a;
0|a|INT|0||0
sqlite> pragma table_info=b;
0|b|INTEGER|0||0
sqlite> pragma table_info=c;
0|c|glauber|0||0

So we'll keep it as well so we can produce the same responses.
2025-01-30 19:53:36 -05:00
Glauber Costa
f1df43633a change type Display implementation to not show null
This is the behavior that things like pragma table_info seem to
expect.
2025-01-30 19:53:36 -05:00
Alex Miller
9ac52b66d9 Refactor cursor to support multiple state machines 2025-01-30 14:08:44 -08:00
Glauber Costa
69d3fbc797 keep track of notnull constraint on column creation 2025-01-30 17:04:12 -05:00
Glauber Costa
42f93e9bea add default type to Column definition 2025-01-30 16:45:57 -05:00
Glauber Costa
7a972318a8 Make query_pragma use enum instead of &str
Fixes #823
2025-01-30 14:06:17 -05:00
krishvishal
6f32344efb Make comparison of type_name case insensitive by converting to uppercase 2025-01-30 17:05:14 +05:30
Pekka Enberg
3a4cb34606 Merge 'Fix memory leaks, make extension types more efficient' from Preston Thorpe
I was baffled previously, because any time that `free` was called on a
type from an extension, it would hang even when I knew it wasn't in use
any longer, and hadn't been double free'd.
After #737 was merged, I tried it again and noticed that it would no
longer hang... but only for extensions that were staticly linked.
Then I realized that we are using a global allocator, that likely wasn't
getting used in the shared library that is built separately that won't
inherit from our global allocator in core, causing some symbol mismatch
and the subsequent hanging on calls to `free`.
This PR adds the global allocator to extensions behind a feature flag in
the macro that will prevent it from being used in `wasm` and staticly
linked environments where it would conflict with limbos normal global
allocator. This allows us to properly free the memory from returning
extension functions over FFI.
This PR also changes the Extension type to a union field so we can store
int + float values inline without boxing them.
any additional tips or thoughts anyone else has on improving this would
be appreciated 👍

Closes #803
2025-01-30 13:31:17 +02:00
Pekka Enberg
c779537f2f Merge 'Strftime compatibility solved' from Pedro Muniz
This PR closes #787. Chrono offers to format the string from an iterator
of Format Items. I created a custom iterator that only allows formatters
specified by sqlite. This approach however does not address the
inefficient way that julianday is calculated. Also, with this
implementation we avoid having to maintain a separate vendored package
for strftime that may become incompatible with Chrono in the future.

Closes #792
2025-01-30 13:30:11 +02:00
Pekka Enberg
e66648beb8 Merge 'Add support for offset in select queries' from Ben Li
#739
Started adding support for `LIMIT...OFFSET...`
- New `OffsetLimit` opcode
- `OFFSET` is now supported for:
    - `SELECT...LIMIT...OFFSET`
    - `SELECT...GROUP BY...LIMIT...OFFSET`
    - `SELECT...ORDER BY...LIMIT...OFFSET`
    - Subqueries for `SELECT` statements
**In progress/todo**
- [x] Testing
- [x] Handle negative offset value
- **(will make in separate PR)** Add support for
`DELETE...LIMIT...OFFSET`
- **(will make in separate PR)** Use `limit + offset` sum register from
`OffsetLimit` to constrain number of records inserted into sorter

Closes #779
2025-01-30 13:29:49 +02:00
Pekka Enberg
5614a7751c Merge 'implement isnull / not null for filter expressions' from Glauber Costa
Allow us to write queries like:
        SELECT name, type, sql FROM sqlite_schema where sql isnull
and
        SELECT name, type, sql FROM sqlite_schema where sql not null

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #829
2025-01-30 13:28:53 +02:00
Pekka Enberg
a2ac3132c4 Merge 'Fix SELECT ABS(-9223372036854775808) causes limbo to panic. ' from Krishna Vishal
Now we return `RuntimeError`.  Matches SQLite behavior.
SQLite:
```sql
sqlite> SELECT ABS(-9223372036854775808);
Runtime error: integer overflow
```
Limbo after this fix:
```sql
limbo> SELECT ABS(-9223372036854775808);
Runtime error: integer overflow
```
Closes https://github.com/tursodatabase/limbo/issues/815

Closes #818
2025-01-30 13:25:39 +02:00
Pekka Enberg
4673ac969e Merge 'Fix SELECT -9223372036854775808 result differs from SQLite' from Krishna Vishal
Closes #812
`-9223372036854775808` is `MIN_INT64`. So when we extract out the minus
and try to parse the remainder it becomes greater than MAX_INT64
(9223372036854775807) and will trigger overflow, which converts the
literal into `Real`. So we have to handle it as a special case.

Reviewed-by: Kim Seon Woo (@seonWKim)

Closes #814
2025-01-30 13:25:27 +02:00
krishvishal
cab0625017 Fixes limbo creating an extra btree, when table has single column PRIMARy KEY.
The error is due to comparing the PRIMARY KEY's name to INTEGER when in it was all in lowercase.
2025-01-30 15:04:50 +05:30
Ihor Andrianov
8a01b842a5 fix function import 2025-01-30 04:05:05 +02:00
Glauber Costa
effde1cc04 implement isnull / not null for filter expressions
Allow us to write queries like:

	SELECT name, type, sql FROM sqlite_schema where sql isnull

and

	SELECT name, type, sql FROM sqlite_schema where sql not null
2025-01-29 20:58:04 -05:00
Ihor Andrianov
d968b314ed fix bug for 1 arg 2025-01-30 03:44:33 +02:00
Ihor Andrianov
ee52192cd8 add unit tests 2025-01-30 03:13:58 +02:00
Ihor Andrianov
ccf51cae80 moved is_json_valid above tests 2025-01-30 02:47:11 +02:00
Ihor Andrianov
5cf80d8cef cargo clippy 2025-01-30 02:40:35 +02:00
Ihor Andrianov
52eab0544a add Target discrete type to handle array and obj changes 2025-01-30 02:09:01 +02:00
Ihor Andrianov
30d810bfe5 add utility function to get mut ref by path 2025-01-30 00:05:53 +02:00
Ihor Andrianov
305e86ec39 allow path parser accept numbers as keys 2025-01-29 22:39:35 +02:00
Ihor Andrianov
d57d9bef6f add function definition 2025-01-29 22:37:04 +02:00
Pekka Enberg
06edf33878 Merge 'json_patch() function implementation' from Ihor Andrianov
First review #820
The function follows RFC 7386 JSON Merge Patch semantics:
* If the patch is null, the target is replaced with null
* If the patch contains a scalar value, the target is replaced with that
value
* If both target and patch are objects, the patch is recursively applied
* null values in the patch result in property removal from the target

Closes #821
2025-01-29 19:54:12 +02:00
Pekka Enberg
db72756d2a Merge 'Changes to json serialization/deserialization' from Ihor Andrianov
Change JSON deserialization to enable json_patch implementation with
SQLite-compatible behavior:
* Preserves duplicate keys in JSON objects
* Applies patches only to the first occurrence of each key
* Trade-off: Changes key lookup from O(1) to O(n) to support duplicate
keys
* Have to be merged before json_patch() function

Closes #820
2025-01-29 19:53:41 +02:00
krishvishal
f8c1828ddf Matched on i64::checked_abs and changed RuntimeError to IntegerOverflow 2025-01-29 22:34:19 +05:30
Pekka Enberg
4af6eb2f71 Merge 'Refactor Json serialization to accommodate formatters for pretty printing' from Pedro Muniz
Json serialization logic was pulled from serde_json. Google's json5
serialization code was not flexible enough to allow for pretty printing
json, so I believe that the formatter design is a good layer to abstract
this logic. This refactor will trivially enable the implementation of
json_pretty function from sqlite. My other PR for json_quote, #763,
depends a tiny bit on a helper utility from the previous serialization
implementation. If this PR is considered first, I will change the code
in my other PR to account for this.

Reviewed-by: Diego Reis (@diegoreis42)
Reviewed-by: Kacper Madej (@madejejej)

Closes #771
2025-01-29 19:03:14 +02:00