Commit Graph

545 Commits

Author SHA1 Message Date
Pekka Enberg
00a546e775 Merge 'Fix string funcs' from Nikita Sivukhin
Add fuzz test for string functions and fix 2 bugs in implementation of
`LTRIM/RTRIM/TRIM` and `QUOTE`:
- `QUOTE` needs to escape internal quote(`'`) symbols
- `LTRIM`/`RTRIM`/`TRIM` needs to check if they have additional argument

Closes #958
2025-02-10 11:17:51 +02:00
Pekka Enberg
55058cade3 Merge 'Fix cast' from Nikita Sivukhin
Fix codegen for `CAST` expression and also adjust implementation of
`CAST: Real -> Int` because `SQLite` use "closest integer between the
REAL value and zero" as a CAST result.

Closes #956
2025-02-10 10:53:56 +02:00
Pekka Enberg
31886e8f4c Merge 'Fix case and emit' from Nikita Sivukhin
(after fixing complicated b-tree bugs - I want to end my day with the
joy of fixing simple bugs)
This PR fix bug in emit (which was introduced after switch from
`HashMap` to `Vec`) and also align `CASE` codegen in case of `NULL`
result from `WHEN` clause with SQLite behaviour.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #953
2025-02-10 10:53:31 +02:00
Nikita Sivukhin
5773f767af fix bug after refactoring in LIKE and GLOB 2025-02-09 23:42:53 +04:00
Nikita Sivukhin
a744dd8419 translate_and_mark helper with its previous signature is error prone - so replace it with more explicit option which accept target register 2025-02-09 23:18:46 +04:00
Nikita Sivukhin
dff1fdc853 fix codegen of CAST 2025-02-09 22:20:28 +04:00
Nikita Sivukhin
a61736bec1 adjust evaluation of NULL results in WHEN clauses with SQLite spec 2025-02-09 21:33:54 +04:00
Aarni Koskela
eaea02c567 Fix a handful of typos 2025-02-09 18:08:29 +02:00
Pekka Enberg
9e65d7ddea Merge 'perf/prepare: box even more stuff to shrink the YYMINORTYPE enum further' from Jussi Saurio
the boxings will continue until morale improves
<img width="621" alt="Screenshot 2025-02-09 at 14 16 21"
src="https://github.com/user-
attachments/assets/887d218d-3db7-4a64-ad81-b7431adb23bb" />

Closes #947
2025-02-09 17:36:03 +02:00
Jussi Saurio
36a3cb1d5e sqlite3-parser: box AlterTable 2025-02-09 14:11:34 +02:00
Jussi Saurio
72a055e5fe sqlite3-parser: box Pragma 2025-02-09 13:10:52 +02:00
Jussi Saurio
23c4106433 sqlite3-parser: separate boxed Insert struct 2025-02-09 13:10:21 +02:00
Jussi Saurio
32887518ce sqlite3-parser: separate boxed Delete struct 2025-02-09 13:10:21 +02:00
Jussi Saurio
40a8dc14cd sqlite3-parser: separate boxed SelectInner struct 2025-02-09 12:54:30 +02:00
Jussi Saurio
358fda2ec7 sqlite3-parser: box the create table body 2025-02-09 12:42:53 +02:00
Pekka Enberg
e406a030e6 Merge 'Rework io_uring feature' from Jorge López Tello
This makes io_uring the default in CLI, but makes it non-default in
core. Before, if one built CLI without io_uring, core still built with
it as it was a default feature. To accommodate for the change, all
bindings have been updated to select the feature, except for WASM which
has a separate fs implementation.
This also adds some #[cfg] and #[allow] to silence unused-* warnings,
which I discovered when testing with different features disabled.

Closes #942
2025-02-09 08:46:29 +02:00
Jorge López
83b158fb3a core: silence some unused warnings when building without default features 2025-02-09 01:13:12 +01:00
Jussi Saurio
781aa3b5d6 sqlite3-parser: box the having clause in GroupBy 2025-02-08 18:10:26 +02:00
Jussi Saurio
670dac5939 sqlite3-parser: box the where clause in Delete 2025-02-08 18:10:25 +02:00
Pekka Enberg
8c0c967ea2 Merge 'Implement json_quote' from Pedro Muniz
Hi! This is my first PR on the project, so I apologize if I did not
follow a convention from the project.
#127
This PR implements json_quote as specified in their source:
https://www.sqlite.org/json1.html#jquote. It follows the internal doc
guidelines for implementing functions. Most tests were added from sqlite
test suite for json_quote, while some others were added by me. Sqlite
test suite for json_quote depends on json_valid to test for correct
escape control characters, so that specific test at the moment cannot be
done the same way.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Reviewed-by: Sonny (@sonhmai)

Closes #763
2025-02-07 13:33:05 +02:00
pedrocarlo
26388cc802 fix: cargo fmt 2025-02-06 23:36:02 -03:00
pedrocarlo
303a687e65 rebase to main 2025-02-06 23:35:58 -03:00
Jussi Saurio
d5f58f5fea Add quickcheck tests for generate_series() and refine implementation 2025-02-06 18:36:21 +02:00
PThorpe92
ae88d51e6f Remove TableReferenceType enum to clean up planner 2025-02-06 09:15:39 -05:00
PThorpe92
d4c06545e1 Refactor vtable impl and remove Rc Refcell from module 2025-02-06 09:15:39 -05:00
Jussi Saurio
f5f77c0bd1 Initial virtual table implementation 2025-02-06 07:51:50 -05:00
Pekka Enberg
f3902ef9b6 core: Rename OwnedRecord to Record
We only have one record type so let's call it `Record`.
2025-02-06 13:40:34 +02:00
krishvishal
32080aba5d Make vector function accessible through Function op code. 2025-02-06 07:01:50 +05:30
Pekka Enberg
6ea7fa06d2 Merge 'prepare perf: make ProgramBuilder aware of plan to count/estimate required memory' from Jussi Saurio
Use knowledge of query plan to inform how much memory to initially
allocate for `ProgramBuilder` vectors
Some of them are exact, some are semi-random estimates
```sql
Prepare `SELECT 1`/Limbo/SELECT 1
                        time:   [756.93 ns 758.11 ns 759.59 ns]
                        change: [-4.5974% -4.3153% -4.0393%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  2 (2.00%) low severe
  1 (1.00%) low mild
  3 (3.00%) high mild
  1 (1.00%) high severe

Prepare `SELECT * FROM users LIMIT 1`/Limbo/SELECT * FROM users LIMIT 1
                        time:   [1.4739 µs 1.4769 µs 1.4800 µs]
                        change: [-7.9364% -7.7171% -7.4979%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou...`
                        time:   [3.7440 µs 3.7520 µs 3.7596 µs]
                        change: [-5.4627% -5.1578% -4.8445%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe
```

Closes #899
2025-02-05 18:24:16 +02:00
Pekka Enberg
b5f5e40986 Merge 'prepare perf: dont eagerly allocate result column name strings' from Jussi Saurio
- Remove eagerly allocated `name` from `ResultSetColumn`
- `ResultSetColumn` can calculate `name()` on demand:
    - if it has an alias (`foo as bar`), use that
    - if it is a column reference, use that
    - otherwise return none, and callers can assign it a placeholder
name (like `column_1`)
- move the `plan.result_columns` and `plan.table_references` to
`Program` after preparing statement is done, so that column names can be
returned upon request
- make `name` in `Column` optional, not needed for pseudo tables and
sorters so avoids an extra string allocation
```sql
Prepare `SELECT 1`/Limbo/SELECT 1
                        time:   [756.80 ns 758.27 ns 760.04 ns]
                        change: [-3.3257% -3.0252% -2.7035%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  2 (2.00%) low severe
  3 (3.00%) low mild
  1 (1.00%) high mild
  2 (2.00%) high severe

Prepare `SELECT * FROM users LIMIT 1`/Limbo/SELECT * FROM users LIMIT 1
                        time:   [1.4646 µs 1.4669 µs 1.4696 µs]
                        change: [-6.4769% -6.2021% -5.9137%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) low severe
  3 (3.00%) low mild
  3 (3.00%) high severe

Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou...`
                        time:   [3.7256 µs 3.7311 µs 3.7376 µs]
                        change: [-4.5195% -4.2192% -3.9309%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) low severe
  2 (2.00%) low mild
  2 (2.00%) high mild
```

Closes #898
2025-02-05 18:20:01 +02:00
Jussi Saurio
795576b2ec dont eagerly allocate result column name strings 2025-02-05 17:53:23 +02:00
Jussi Saurio
f599b5a752 Make programbuilder aware of plan to count/estimate required memory 2025-02-05 14:22:42 +02:00
Pekka Enberg
56d401fb67 Merge 'Implement json_set' from Marcus Nilsson
This PR adds support for `json_set`.
There are three helper functions added:
1. `json_path_from_owned_value`, this function turns an `OwnedValue`
into a `JsonPath`.
2. `find_or_create_target`, this function is similar to `find_target`
with the added bonus of creating the target if it doesn't exist. There
is a caveat with this function and that is that it will create
objects/arrays as it goes, meaning if you send `{}` into it and try
getting the path `$.some.nested.array[123].field`, it will return
`{"some":{"nested":array:[]}}` since creation of `some`, `nested` and
`array` will succeed, but accessing element `123` will fail.
3. `create_and_mutate_json_by_path`, this function is very similar to
`mutate_json_by_path` but calls `find_or_create_target` instead of
`find_target`

Related to #127

Closes #878
2025-02-05 14:15:02 +02:00
Pekka Enberg
9fdf54de2b Merge 'Small perf optimizations to statement preparation' from Jussi Saurio
```bash
Prepare `SELECT 1`/Limbo/SELECT 1
                        time:   [765.94 ns 768.26 ns 771.03 ns]
                        change: [-7.8340% -7.4887% -7.1406%] (p = 0.00 < 0.05)
                        Performance has improved.

Prepare `SELECT * FROM users LIMIT 1`/Limbo/SELECT * FROM users LIMIT 1
                        time:   [1.5673 µs 1.5699 µs 1.5731 µs]
                        change: [-10.810% -9.7122% -8.4951%] (p = 0.00 < 0.05)
                        Performance has improved.

Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou...
                        time:   [4.1331 µs 4.1421 µs 4.1513 µs]
                        change: [-9.3157% -9.0255% -8.7372%] (p = 0.00 < 0.05)
                        Performance has improved.
```
flamegraph for prepare `SELECT 1`:
<img width="1718" alt="Screenshot 2025-02-03 at 10 34 14"
src="https://github.com/user-
attachments/assets/ba67fe2f-78b2-4796-9a09-837d8e79fe62" />

Closes #872
2025-02-05 10:46:57 +02:00
Marcus Nilsson
01492cf46f add support for json_set
Test cases are included.
Related to #127
2025-02-04 19:09:58 +01:00
Pekka Enberg
bf1ef13c91 Merge 'Add Printf Support' from Zaid Humayun
Add basic printf function support in limbo
![Screenshot 2025-02-04 at 8 08 23 PM](https://github.com/user-
attachments/assets/b12931eb-8e79-4c8a-af77-c25c34cc5834)

Closes #886
2025-02-04 17:53:27 +02:00
Zaid Humayun
6a863b3da9 printf: this commit adds support for https://github.com/tursodatabase/limbo/issues/885 tracking printf functionality
this commit introduces basic support for printf functionality and doesn't include advanced modifiers like width etc.
2025-02-04 20:05:14 +05:30
Pekka Enberg
2d9a54ccba Merge 'implement json_pretty' from Pedro Muniz
This PR implements json_pretty. At the moment, support for jsonb is
being added, so this function suffers from the same limitations as in
json(x). Also, I have not found a way to implement the same conversion
of Blob -> String that SQLite does. From my own experimentation, I
believe SQLite converts blobs to a lossy ascii representation, but I
would appreciate some help on this.

Closes #860
2025-02-04 14:51:47 +02:00
Pekka Enberg
0050f4aeea Merge 'Support column aliases in GROUP BY, ORDER BY and HAVING' from Jussi Saurio
Closes #744
```sql
# Wanda = 9, Whitney = 11, William = 111
do_execsql_test column_alias_in_group_by_order_by_having {
  select first_name as fn, count(1) as fn_count from users where fn in ('Wanda', 'Whitney', 'William') group by fn having fn_count > 10 order by fn_count;
} {Whitney|11
William|111}
```

Closes #864
2025-02-04 14:51:05 +02:00
Jussi Saurio
d182ddf514 dont store insn comments unless the query is EXPLAIN 2025-02-03 19:53:33 +02:00
Jussi Saurio
750a9c6463 assertions and small cleanups 2025-02-03 13:08:13 +02:00
Jussi Saurio
40f536fabb Dont store available_indexes on plan; only used in optimize_plan() 2025-02-03 12:52:14 +02:00
Jussi Saurio
61a007fb29 preallocate plan.result_columns according to AST 2025-02-03 12:51:24 +02:00
Jussi Saurio
f2dab8499d preallocate loop metadata according to table/column count and prefer vec over hashmap 2025-02-03 12:51:24 +02:00
Jussi Saurio
d4cb0a1223 Merge 'Fix logical codegen' from Nikita Sivukhin
Fix few logical codegen issues and add fuzz tests for logical
expressions
-  Right now Limbo fails to recognize `false` constant in case when any
unary operator is used on the AST path. This PR add unary operator
option in the rewrite code and handle such cases properly.
```sql
limbo> SELECT NOT FALSE;

  × Parse error: no such column: FALSE - should this be a string literal in single-quotes?

```
- `ifnull` implementation produced incorrect codegen due to "careless"
management of registers
```
limbo> SELECT ifnull(0, NOT 0)
[NULL here]
```
- `like` implementation produced incorrect codegen due to "careless"
management of registers
```
limbo> SELECT like('a%', 'a') = 1;
thread 'main' panicked at core/vdbe/mod.rs:1902:41:
internal error: entered unreachable code: Like on non-text registers
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
Depends on https://github.com/tursodatabase/limbo/pull/867 (need
`GrammarGenerator` from this branch)

Closes #869
2025-02-03 12:39:41 +02:00
Jussi Saurio
1e5501650a Support column aliases in GROUP BY, ORDER BY and HAVING 2025-02-03 10:44:05 +02:00
Nikita Sivukhin
979612cb34 fix miscompilation of like function 2025-02-03 11:25:14 +04:00
Nikita Sivukhin
11c47f5e44 fix miscomplation of ifnull scalar function 2025-02-03 11:25:14 +04:00
Nikita Sivukhin
a4a80f37bc rewrite unary expressions too - in order to support "NOT FALSE" expressions 2025-02-03 11:25:14 +04:00
Nikita Sivukhin
5a3587f7a2 use opposite operator for search if WHERE condition is swapped (e.g. 1 > x instead of x < 1) 2025-02-03 11:23:04 +04:00