Commit Graph

4687 Commits

Author SHA1 Message Date
Jussi Saurio
fbfd2b2c38 refactor: use walk_expr_mut() in rewrite_expr() 2025-05-23 16:27:28 +03:00
Jussi Saurio
362347c474 refactor: use walk_expr() in determine_where_to_eval_expr() 2025-05-23 16:27:28 +03:00
Jussi Saurio
9ec84e3905 refactor: use walk_expr() in table_mask_from_expr() 2025-05-23 16:27:28 +03:00
Jussi Saurio
3835a29f47 refactor: use walk_expr() in resolve_aggregates() 2025-05-23 16:27:23 +03:00
Jussi Saurio
2ab5c5f6a9 refactor: use walk_expr_mut() in bind_column_references() 2025-05-23 15:56:49 +03:00
Jussi Saurio
40a4d162bc Introduce walker expressions for ast::Expr 2025-05-23 15:56:27 +03:00
Jussi Saurio
597020bc0c Merge 'Support values statement and values in select' from meteorgan
Close: #866
**limbo output**:
```
limbo> explain values(1, 2);
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     5     0                    0   Start at 5
1     Integer            1     1     0                    0   r[1]=1
2     Integer            2     2     0                    0   r[2]=2
3     ResultRow          1     2     0                    0   output=r[1..2]
4     Halt               0     0     0                    0
5     Goto               0     1     0                    0

limbo> explain values(1, 2), (3, 4);
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     16    0                    0   Start at 16
1     InitCoroutine      1     9     2                    0
2     Integer            1     2     0                    0   r[2]=1
3     Integer            2     3     0                    0   r[3]=2
4     Yield              1     0     0                    0
5     Integer            3     2     0                    0   r[2]=3
6     Integer            4     3     0                    0   r[3]=4
7     Yield              1     0     0                    0
8     EndCoroutine       1     0     0                    0
9     InitCoroutine      1     0     2                    0
10    Yield              1     15    0                    0
11    Copy               2     4     0                    0   r[4]=r[2]
12    Copy               3     5     0                    0   r[5]=r[3]
13    ResultRow          4     2     0                    0   output=r[4..5]
14    Goto               0     10    0                    0
15    Halt               0     0     0                    0
16    Goto               0     1     0                    0

limbo> explain select * from (values(1, 2), (3, 4));
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     16    0                    0   Start at 16
1     InitCoroutine      1     9     2                    0
2     Integer            1     2     0                    0   r[2]=1
3     Integer            2     3     0                    0   r[3]=2
4     Yield              1     0     0                    0
5     Integer            3     2     0                    0   r[2]=3
6     Integer            4     3     0                    0   r[3]=4
7     Yield              1     0     0                    0
8     EndCoroutine       1     0     0                    0
9     InitCoroutine      1     0     2                    0
10    Yield              1     15    0                    0
11    Copy               2     4     0                    0   r[4]=r[2]
12    Copy               3     5     0                    0   r[5]=r[3]
13    ResultRow          4     2     0                    0   output=r[4..5]
14    Goto               0     10    0                    0
15    Halt               0     0     0                    0
16    Transaction        0     0     0                    0   write=false
17    Goto               0     1     0                    0
```
**sqlite output**:
```
sqlite> explain values(1, 2);
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     5     0                    0   Start at 5
1     Integer        1     1     0                    0   r[1]=1
2     Integer        2     2     0                    0   r[2]=2
3     ResultRow      1     2     0                    0   output=r[1..2]
4     Halt           0     0     0                    0
5     Goto           0     1     0                    0
sqlite> explain values(1, 2), (3, 4);
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     16    0                    0   Start at 16
1     InitCoroutine  1     9     2                    0
2     Integer        1     4     0                    0   r[4]=1
3     Integer        2     5     0                    0   r[5]=2
4     Yield          1     0     0                    0
5     Integer        3     4     0                    0   r[4]=3
6     Integer        4     5     0                    0   r[5]=4
7     Yield          1     0     0                    0
8     EndCoroutine   1     0     0                    0
9     InitCoroutine  1     0     2                    0
10      Yield          1     15    0                    0   next row of 2-ROW VALUES CLAUSE
11      Copy           4     8     0                    2   r[8]=r[4]
12      Copy           5     9     0                    2   r[9]=r[5]
13      ResultRow      8     2     0                    0   output=r[8..9]
14    Goto           0     10    0                    0
15    Halt           0     0     0                    0
16    Goto           0     1     0                    0
sqlite>  explain select * from (values(1, 2), (3, 4));
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     16    0                    0   Start at 16
1     InitCoroutine  1     9     2                    0
2     Integer        1     4     0                    0   r[4]=1
3     Integer        2     5     0                    0   r[5]=2
4     Yield          1     0     0                    0
5     Integer        3     4     0                    0   r[4]=3
6     Integer        4     5     0                    0   r[5]=4
7     Yield          1     0     0                    0
8     EndCoroutine   1     0     0                    0
9     InitCoroutine  1     0     2                    0
10      Yield          1     15    0                    0   next row of 2-ROW VALUES CLAUSE
11      Copy           4     8     0                    2   r[8]=r[4]
12      Copy           5     9     0                    2   r[9]=r[5]
13      ResultRow      8     2     0                    0   output=r[8..9]
14    Goto           0     10    0                    0
15    Halt           0     0     0                    0
16    Goto           0     1     0                    0
```

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1549
2025-05-23 13:56:31 +03:00
meteorgan
01c8a4ca63 simpify values when it's subquery 2025-05-23 17:45:56 +08:00
Jussi Saurio
128a406f8c TableReference: fix stale comment 2025-05-23 10:06:24 +03:00
Jussi Saurio
89a727022a Merge 'Drop Table OpCodes Use Ephemeral Table As Scratch Table' from Zaid Humayun
In SQLite, when [AUTOVACUUM](https://www.sqlite.org/lang_vacuum.html) is
enabled (in SQLite, the `SQLITE_OMIT_AUTOVACUUM` flag is used), and a
table is dropped, if the root page of the table is destroyed and *it is
not the last page*, then the last root page is transferred here to avoid
empty gaps.
At this point, the old root page value is copied into a register and
SQLite will copy over all rows using the old root page from the schema
table into an ephemeral table. Later, it copies over the rows from the
ephemeral table back into the schema table with the new root page number
to "patch" the row.
This PR introduces the same semantics to Limbo. It doesn't perform the
AUTOVACUUM functionality of moving the last root page into the empty
page currently, so it no-ops that by returning 0. This triggers an
`IfNot` instruction to skip the inserts into the ephemeral table.
I plan to address the auto vacuuming in a future PR. But this brings us
a lot closer to matching SQLite semantics.
![Screenshot 2025-05-21 at 8 18 12 PM](https://github.com/user-
attachments/assets/049125ad-4448-4c14-b647-8ed990aaadaf)

Closes #1548
2025-05-23 09:46:30 +03:00
Zaid Humayun
bc5d93c18a Addresses comment https://github.com/tursodatabase/limbo/pull/1548#discussion_r2103333264 by @jussisaurio
this commit uses more descriptive names for registers
2025-05-23 10:19:08 +05:30
Zaid Humayun
4312d371fb addresses comment https://github.com/tursodatabase/limbo/pull/1548#discussion_r2102606810 by @jussisaurio
this commit changes the btree_destroy() signature to return an Option<usize>. This more closely resembles Rust semantics instead of passing a pointer to a usize.
However, I'm unsure if I'm handling the cursor result correctly
2025-05-23 00:46:05 +05:30
Zaid Humayun
780d0cb5a8 addresses https://github.com/tursodatabase/limbo/pull/1548#discussion_r2102599487, https://github.com/tursodatabase/limbo/pull/1548#discussion_r2102601189 & https://github.com/tursodatabase/limbo/pull/1548#discussion_r2102602774 by @jussisaurio
this commit addresses comments regarding using decsriptive variable names for the loops and loop labels. Also, adds documentation for instructions that cause jumps in both loops
2025-05-23 00:31:19 +05:30
Zaid Humayun
3a8b18c481 addressed comment https://github.com/tursodatabase/limbo/pull/1548#discussion_r2102595246 by @jussisaurio
this commit addresses the comment regarding inline code documentation
2025-05-23 00:12:27 +05:30
Zaid Humayun
1999eac891 Fixes test Testing: can drop kv_store vtable
Earlier this test broke because the code to translate the drop table was not checking to see if a table was not a virtual table.
SQLite does this with a macro called IsVirtualTable. Here, I check with the existing methods on the BTree struct
2025-05-22 23:53:48 +05:30
meteorgan
34e05ef974 make values work in subquery 2025-05-23 00:30:04 +08:00
meteorgan
0467d7e11b Support values statement and values in select 2025-05-23 00:29:54 +08:00
Zaid Humayun
4072a41c9c Drop Table now uses an ephemeral table as a scratch table
Now when dropping a table, an ephemeral table is created as a scratch table. If a root page of some other table is moved into the page occupied by the root page of the table being dropped, that row is first written into an ephemeral table. Then on a next pass, it is deleted from the schema table and then re-inserted with the new root page.

This happens during AUTOVACUUM when deleting a root page will force the last root page to move into the slot being vacated by the root page of the table being deleted
2025-05-22 19:39:46 +05:30
Jussi Saurio
0c4c451d2a rename 2025-05-22 16:51:03 +03:00
Jussi Saurio
6ed5412bde extract method 2025-05-22 16:51:03 +03:00
Jussi Saurio
df8a19767f Fixes to account for collation 2025-05-22 16:51:03 +03:00
Jussi Saurio
afc94cd3be Add basic select distinct TCL test 2025-05-22 16:51:03 +03:00
Jussi Saurio
f3ea9a603a add support for SELECT DISTINCT 2025-05-22 16:51:03 +03:00
Jussi Saurio
b0c3483e94 Allocate ephemeral index for SELECT DISTINCT 2025-05-22 16:51:03 +03:00
Jussi Saurio
76227ec274 Rename to Distinctness + add distinctness information to SelectPlan 2025-05-22 16:51:03 +03:00
Pekka Enberg
268b30deee Merge 'sqlite3-parser: Remove scanner trace-logging' from Pekka Enberg
It spams the logs like no tomorrow, and is mostly useless.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1555
2025-05-22 13:02:49 +03:00
Pekka Enberg
9f5904d4df cargo fmt 2025-05-22 13:02:30 +03:00
Pekka Enberg
5a9aa5a119 Merge 'sqlite3: Switch to tracing logger' from Pekka Enberg
...we now actually see logging from core too.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1554
2025-05-22 13:02:24 +03:00
Pekka Enberg
5ed187ba61 sqlite3-parser: Remove scanner trace-logging
It spams the logs like no tomorrow, and is mostly useless.
2025-05-22 12:37:28 +03:00
Pekka Enberg
7d471889eb sqlite3: Switch to tracing logger
...we now actually see logging from core too.
2025-05-22 12:37:09 +03:00
Jussi Saurio
c1e31b0213 Merge 'Fix labeler labeling everything as Extensions-Other' from Jussi Saurio
Closes #1551
2025-05-22 12:20:20 +03:00
Jussi Saurio
e398755667 Merge 'Fix bug in op_decr_jump_zero()' from Jussi Saurio
Closes #1550
2025-05-22 12:20:02 +03:00
Jussi Saurio
4c67bdd266 Fix labeler labeling everything as Extensions-Other 2025-05-22 12:03:28 +03:00
Jussi Saurio
533a00eae3 Fix bug in op_decr_jump_zero() 2025-05-22 11:40:49 +03:00
Jussi Saurio
8bec75d804 Merge 'Initial Support for Nested Translation' from Pedro Muniz
This PR introduces some modifications to the Program Builder to allow us
to use nested parsing. By focusing the emission of Init and the last
Goto (prologue and epilogue), inside the ProgramBuilder, we can just not
emit them if we are parsing/translating in a nested context. For this
PR, I only migrated insert to use these functions as I need them to
support Insert statements that use `SELECT FROM` syntax. Nested parsing
overall enables code reuse for us and arguably is one of the only ways
to parse deeply nested queries without a lot of code duplication.
#1528

Closes #1543
2025-05-22 10:52:00 +03:00
Jussi Saurio
c7f984c5c8 Merge 'Page cache fixes' from Pere Diaz Bou
This PR builds on top of
https://github.com/tursodatabase/limbo/pull/1368 and adds few things
like allowing inserting pages with the same page key, fix fuzz tests by
adding transactions and some minor improvements to cacheflush.

Closes #1523
2025-05-22 10:12:56 +03:00
Jussi Saurio
fc150b12c9 Merge 'CSV virtual table extension' from Piotr Rżysko
This PR adds a port of [SQLite's CSV virtual table
extension](https://www.sqlite.org/csv.html).
Planned follow-ups:
* Pass detailed error messages from `VTabModule::create`, not just
`ResultCode`s.
* Address the TODO in `VTabModuleImpl::create_schema`.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1544
2025-05-22 09:48:53 +03:00
Jussi Saurio
58bff43f1f Merge 'remove detection of comments in the middle of query in cli' from Pedro Muniz
I was trying to run the TPC-H 9.sql in the CLI, but it kept truncating
my input due to a comment that is present in the query. After removing
it, I can just copy and paste the query and it works. @PThorpe92 is it
safe to remove this? Or is there a particular reason that was included
that I am not aware of?

Closes #1525
2025-05-22 09:48:13 +03:00
Jussi Saurio
a98f315cbc Merge 'cli/fix: Apply default config for app' from Diego Reis
Fix #1546

Closes #1547
2025-05-22 09:44:06 +03:00
pedrocarlo
53bf5d5ef5 adjust translate functions to take a program instead of Option<ProgramBuilder> + remove any Init emission in traslate functions + use epilogue in all places necessary 2025-05-21 16:41:10 -03:00
pedrocarlo
1c12535d9f push prologue to top-level translate function 2025-05-21 15:50:43 -03:00
pedrocarlo
8084d54c26 lift pragma statement handling as it cannot be created in a nested context 2025-05-21 14:13:28 -03:00
pedrocarlo
d21229d4a3 create inner translate function to enable calling it from a nested context 2025-05-21 14:08:02 -03:00
pedrocarlo
3090dd91fa push translate_ctx creation outside of prologue 2025-05-21 13:06:25 -03:00
pedrocarlo
fc08f786fc use prologue and epilogue in insert 2025-05-21 12:47:51 -03:00
pedrocarlo
f5d6d11d16 extract prologue and epilogue to program builder 2025-05-21 12:47:51 -03:00
pedrocarlo
517c7c81cd refactor to include optional program builder argument 2025-05-21 12:47:51 -03:00
Diego Reis
c76387ec1b cli/fix: Apply default config for app 2025-05-21 12:06:08 -03:00
Pere Diaz Bou
b135bf449f reduce attempts for fuzz_long overflow 2025-05-21 15:40:42 +02:00
Pere Diaz Bou
7143e43dd4 clippy 2025-05-21 15:27:15 +02:00