Commit Graph

131 Commits

Author SHA1 Message Date
pedrocarlo
655ceeca45 correct count implementation 2025-05-10 22:23:01 -03:00
Jussi Saurio
37097e01ae GROUP BY: refactor logic to support cases where no sorting is needed 2025-05-08 12:39:26 +03:00
Jussi Saurio
330fedbc2f Add notion of join ordering to plan + make determining where to eval expr dynamic always 2025-05-03 15:32:06 +03:00
Jussi Saurio
3798b4aa8b use SortOrder in sorters always 2025-04-24 10:34:06 +03:00
Jussi Saurio
5a1cfb7d15 Add ColumnUsedMask struct to TableReference to track columns referenced in query 2025-04-15 15:13:31 +03:00
Jussi Saurio
457bded14d optimizer: refactor optimizer to support multicolumn index scans 2025-04-10 15:53:02 +03:00
PThorpe92
13e084351d Change parse_limit function to accept reference value to ast::Limit 2025-04-04 12:38:18 -04:00
Jussi Saurio
eca196a54b Support numeric column references in GROUP BY 2025-02-14 16:57:45 +02:00
Pekka Enberg
ac54c35f92 Switch to workspace dependencies
...makes it easier to specify a version, which is needed for `cargo publish`.
2025-02-12 17:28:04 +02:00
Pekka Enberg
5205c23eed Merge 'Initial support for WITH clauses (common table expressions)' from Jussi Saurio
Adds initial limited support for CTEs.
- No MATERIALIZED
- No RECURSIVE
- No named CTE columns
- Only SELECT statements supported inside CTE
Basically this kind of WITH clause can just be rewritten as a subquery,
so this PR adds some plumbing to rewrite them using the existing
subquery machinery.
It also introduces the concept of a `Scope` where a child query can
refer to its parent, useful for CTEs like:
```
do_execsql_test nested-subquery-cte {
    with nested_sub as (
        select concat(name, '!!!') as loud_hat
        from products where name = 'hat'
    ),
    sub as (
        select upper(nested_sub.loud_hat) as loudest_hat from nested_sub
    )
    select sub.loudest_hat from sub;
} {HAT!!!}
```
I think we need to expand the use of `Scope` to all of our identifier
resolutions (currently we don't explicitly have logic for determining
what a given query can see), but I didn't want to bloat the PR too much.
Hence, this implementation is probably full of all sorts of bugs, but
I've added equivalent tests for ALL the existing subquery tests,
rewritten in CTE form.

Closes #920
2025-02-10 12:15:07 +02:00
Jussi Saurio
40a8dc14cd sqlite3-parser: separate boxed SelectInner struct 2025-02-09 12:54:30 +02:00
Jussi Saurio
781aa3b5d6 sqlite3-parser: box the having clause in GroupBy 2025-02-08 18:10:26 +02:00
Jussi Saurio
9e70e8fe02 Add basic CTE support 2025-02-08 14:50:05 +02:00
PThorpe92
d4c06545e1 Refactor vtable impl and remove Rc Refcell from module 2025-02-06 09:15:39 -05:00
Pekka Enberg
6ea7fa06d2 Merge 'prepare perf: make ProgramBuilder aware of plan to count/estimate required memory' from Jussi Saurio
Use knowledge of query plan to inform how much memory to initially
allocate for `ProgramBuilder` vectors
Some of them are exact, some are semi-random estimates
```sql
Prepare `SELECT 1`/Limbo/SELECT 1
                        time:   [756.93 ns 758.11 ns 759.59 ns]
                        change: [-4.5974% -4.3153% -4.0393%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  2 (2.00%) low severe
  1 (1.00%) low mild
  3 (3.00%) high mild
  1 (1.00%) high severe

Prepare `SELECT * FROM users LIMIT 1`/Limbo/SELECT * FROM users LIMIT 1
                        time:   [1.4739 µs 1.4769 µs 1.4800 µs]
                        change: [-7.9364% -7.7171% -7.4979%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou...`
                        time:   [3.7440 µs 3.7520 µs 3.7596 µs]
                        change: [-5.4627% -5.1578% -4.8445%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe
```

Closes #899
2025-02-05 18:24:16 +02:00
Jussi Saurio
795576b2ec dont eagerly allocate result column name strings 2025-02-05 17:53:23 +02:00
Jussi Saurio
f599b5a752 Make programbuilder aware of plan to count/estimate required memory 2025-02-05 14:22:42 +02:00
Pekka Enberg
9fdf54de2b Merge 'Small perf optimizations to statement preparation' from Jussi Saurio
```bash
Prepare `SELECT 1`/Limbo/SELECT 1
                        time:   [765.94 ns 768.26 ns 771.03 ns]
                        change: [-7.8340% -7.4887% -7.1406%] (p = 0.00 < 0.05)
                        Performance has improved.

Prepare `SELECT * FROM users LIMIT 1`/Limbo/SELECT * FROM users LIMIT 1
                        time:   [1.5673 µs 1.5699 µs 1.5731 µs]
                        change: [-10.810% -9.7122% -8.4951%] (p = 0.00 < 0.05)
                        Performance has improved.

Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou...
                        time:   [4.1331 µs 4.1421 µs 4.1513 µs]
                        change: [-9.3157% -9.0255% -8.7372%] (p = 0.00 < 0.05)
                        Performance has improved.
```
flamegraph for prepare `SELECT 1`:
<img width="1718" alt="Screenshot 2025-02-03 at 10 34 14"
src="https://github.com/user-
attachments/assets/ba67fe2f-78b2-4796-9a09-837d8e79fe62" />

Closes #872
2025-02-05 10:46:57 +02:00
Jussi Saurio
40f536fabb Dont store available_indexes on plan; only used in optimize_plan() 2025-02-03 12:52:14 +02:00
Jussi Saurio
61a007fb29 preallocate plan.result_columns according to AST 2025-02-03 12:51:24 +02:00
Jussi Saurio
1e5501650a Support column aliases in GROUP BY, ORDER BY and HAVING 2025-02-03 10:44:05 +02:00
Jussi Saurio
2ddac4bf21 select.rs: use new data structures when parsing select 2025-02-02 10:18:13 +02:00
ben594
d03a0dbd39 Added parsing of offset clause 2025-01-26 16:40:30 -05:00
Pekka Enberg
f5e5428d45 Merge 'Syntactic improvements' from Jorge López Tello
This is a purely syntactic PR. It doesn't change behavior, just rewrites
some loops and removes unneeded parts, like lifetime annotations and
references. Mainly because the Clippy and IDE warnings get annoying.
Don't worry about the number of commits, I just separated based on type
of change.

Closes #732
2025-01-19 12:17:28 +02:00
Pekka Enberg
cdcc98540a cargo fmt 2025-01-19 08:52:01 +02:00
Krishna Vishal
5cf78b7d54 chore: clippy remove unused imports 2025-01-19 07:18:31 +05:30
Krishna Vishal
acad562c07 Remove unnecessary Result 2025-01-19 06:50:00 +05:30
Krishna Vishal
fa0503f0ce 1. Changes to extension.py
2. chore: cargo fmt
2025-01-19 04:58:05 +05:30
Krishna Vishal
6173aeeb3b 1. Fix merge conflicts
2. change tests for extensions to return error instead of null (Preston)
2025-01-19 04:39:25 +05:30
Krishna Vishal
ca097b1972 Remove unused import 2025-01-19 04:35:21 +05:30
Krishna Vishal
027803aabf Refactor code 2025-01-19 04:35:21 +05:30
Krishna Vishal
68553904c7 Converted the unconditional unwrap to a match which handles the case when the function is COUNT and args are None and replaces the args. Solves https://github.com/tursodatabase/limbo/issues/725 2025-01-19 04:35:21 +05:30
Jorge López
86a4714711 syntactic changes: remove unneeded paths when the type is already imported 2025-01-18 18:29:12 +01:00
PThorpe92
0c737d88f7 Support aggregate functions in Extensions 2025-01-17 14:13:57 -05:00
PThorpe92
9c208dc866 Add tests for first extension 2025-01-14 07:27:35 -05:00
PThorpe92
98eff6cf7a Enable passing arguments to external functions 2025-01-14 07:20:50 -05:00
PThorpe92
3412a3d4c2 Rough design for extension api/draft extension 2025-01-14 07:20:48 -05:00
Levy A.
eff5de50c5 refactor: make translate_* functions accept ProgramBuilder
simplifies function signatures and allows attaching more context to
ProgramStatus on `translate::translate`, useful for query parameters.
2025-01-13 20:41:56 -03:00
Jussi Saurio
f434b24e63 Fix limbo/core to work with new boxed ast types 2025-01-05 13:51:34 +02:00
Jussi Saurio
2b5b54c44e clippy 2025-01-01 07:56:39 +02:00
Jussi Saurio
2066475e03 feat: subqueries in FROM clause 2024-12-31 14:18:29 +02:00
Pekka Enberg
33dbd6c892 core: External functions 2024-12-31 13:56:32 +02:00
Pekka Enberg
cb5d86ed8e core/translate: Move prepare_select_plan() to select.rs
The planner.rs file is pretty big. Let's make it smaller by moving more
of select handling to select.rs.
2024-12-31 11:38:13 +02:00
김선우
5cdcb8d78c Split Plan into Select and Delete 2024-12-23 05:45:23 +09:00
김선우
1d3ce52812 Refactor planner and optimizer to be DRY 2024-12-22 15:11:26 +09:00
jussisaurio
7ecc252507 fix rest of the failing tests 2024-11-26 17:31:51 +02:00
jussisaurio
3f9e60633f select refactor: order by and basic agg kinda work 2024-11-26 17:31:51 +02:00
jussisaurio
d0466e1cae introduce Column member of ast::Expr and bind idents to columns 2024-11-26 17:31:51 +02:00
Pere Diaz Bou
c0e51c4ca6 wip wal 2024-11-05 15:29:53 +01:00
jussisaurio
a108dea825 GROUP BY 2024-09-14 16:14:45 +03:00