Pekka Enberg
edf0f754f6
Merge 'More structured query planner' from Jussi Saurio
...
Reader's guide to this PR:
The aim is to have a more structured and maintainable approach to generating bytecode from the query AST so that different parts of the query processing pipeline have clearer responsibilities, so that developing new functionality is easier. E.g.:
- If you want to implement join reordering -> you do it in `Optimizer`
- If you want to implement `GROUP BY` -> you change `QueryPlanNode::Aggregate` to include it, parse it in `Planner` and handle the code generation for it in `Emitter`
The pipeline is:
`SQL text -> Parser -> Planner -> Optimizer -> Emitter`
and this pipeline generates:
`SQL text -> AST -> Logical Plan -> Optimized Logical Plan -> SQLite Bytecode`
---
Module structure:
`plan.rs`: defines the `Operator` enum. An `Operator` is a tree of other `Operators`, e.g. an `Operator::Join` has `left` and `right` children, etc.
`planner.rs`: Parses an `ast::Select` into a `Plan` which is mainly a wrapper for a root `Operator`
`optimizer.rs`: Makes a new `Plan` from an input `Plan` - does predicate pushdown, constant elimination and turns `Scan` nodes into `SeekRowId` nodes where applicable
`emitter.rs`: Generates bytecode instructions from an input `Plan`.
---
Adds feature `EXPLAIN QUERY PLAN <stmt>` which shows the logical query plan instead of the bytecode plan
---
Other changes:
- Almost everything from `select.rs` removed; things like `translate_aggregation()` moved to `expr.rs`
- `where_clause.rs` removed, some things from it like `translate_condition_expr()` moved to `expr.rs`
- i.e.: there is nothing _new_ in `expr.rs`, stuff just moved there
---
Concerns:
- Perf impact: there's a lot more indirection than before (`Operator`s are very "traditional" trees where they refer to other operators via Boxes etc)
Closes #281
2024-08-18 16:36:51 +03:00
jussisaurio
a79c0c5b34
BytecodeGenerator struct was unnecessary
2024-08-17 14:35:44 +03:00
jussisaurio
2b71a5802d
tweak
2024-08-17 14:24:20 +03:00
jussisaurio
b7fbe57ca7
Move translate_table_columns to expr.rs
2024-08-17 14:16:56 +03:00
jussisaurio
05a6616803
BytecodeGenerator struct
2024-08-17 14:12:57 +03:00
jussisaurio
d7d195a618
Cleanup and improve emitter.rs docs
2024-08-17 13:56:59 +03:00
jussisaurio
e8c894e532
More flexible Emitter via stateful operators
2024-08-17 12:55:16 +03:00
JeanArhancet
b6c720c90a
feat: add quote function
2024-08-17 09:28:14 +02:00
jussisaurio
d70eb6b3d7
fix seekrowid operator not emitting result when root
2024-08-17 09:16:24 +03:00
jussisaurio
69f549d2b9
remove unnecessary unwrap_or
2024-08-17 09:07:29 +03:00
jussisaurio
2e7f240bb5
use table_columns() for seekrowid result_columns()
2024-08-17 09:04:27 +03:00
jussisaurio
25033d280c
more accurate variable name
2024-08-16 19:58:59 +03:00
jussisaurio
17cc3717c8
rebase
2024-08-16 19:43:29 +03:00
jussisaurio
069826820e
Finish renaming node -> operator
2024-08-16 19:42:03 +03:00
jussisaurio
97dc98336c
fix comment
2024-08-16 19:42:03 +03:00
jussisaurio
e7cc04e157
Operator comments
2024-08-16 19:42:03 +03:00
jussisaurio
4c016b042b
comment about bitmasks
2024-08-16 19:42:03 +03:00
jussisaurio
1130ccf203
mutable out parameter
2024-08-16 19:42:03 +03:00
jussisaurio
9ab08ee2e6
is_rowid_alias instead of primary_key
2024-08-16 19:42:03 +03:00
jussisaurio
2e32ca0bdb
More structured query planner
2024-08-16 19:42:03 +03:00
Kim Seon Woo
48d3c05fb0
Rebase
2024-08-16 21:40:30 +09:00
Pekka Enberg
c9c0d2808c
Merge 'Add support for ifnull scalar function' from Kim Seon Woo
...
Add support for `ifnull` scalar function
### EXPLAIN SELECT ifnull(null, 20);
### Related issue
https://github.com/penberg/limbo/issues/144
Closes #290
2024-08-16 14:37:08 +03:00
Pekka Enberg
ba3acedbc8
Merge 'Add support for substr scalar function' from Kim Seon Woo
...
Add support for `substr` scalar function. We can reuse the `substring` logic which is already implemted.
## Related issue
https://github.com/penberg/limbo/issues/144
Closes #289
2024-08-16 14:36:55 +03:00
김선우
8fbcd10e0e
Add support for ifnull scalar function
2024-08-16 09:25:04 +09:00
김선우
27d9af2bf4
Add support for substr scalar function
2024-08-16 08:26:42 +09:00
김선우
fe526e118e
Nit
2024-08-16 06:43:55 +09:00
김선우
ced664621f
Allow char function to have empty args
2024-08-16 06:33:18 +09:00
김선우
77c3d130f3
Add char function support
2024-08-16 06:26:06 +09:00
Pekka Enberg
39e77bac9f
Merge 'Implement scala function substring(...) ' from Kim Seon Woo
...
Implement scala function `substring(x, y, z)` and `substring(x, y)`
### EXPLAIN SELECT substring('limbo', 3);
### EXPLAIN SELECT substring('limbo', 1, 2);
### Related issue
https://github.com/penberg/limbo/issues/144
Closes #286
2024-08-15 19:45:22 +03:00
김선우
ac440495bd
Nit
2024-08-15 18:59:46 +09:00
김선우
28bd0275c0
Run cargo fmt, clippy
2024-08-15 18:56:02 +09:00
김선우
aff13de125
Nit
2024-08-15 18:50:40 +09:00
김선우
dc48c2e71d
Add substring scala function with 2 arg
2024-08-15 18:47:32 +09:00
김선우
3fa3c5c002
Add substr scala function
2024-08-15 18:27:18 +09:00
Vegard Stikbakke
e54fd83f49
Handle FunctionCallStar in analyze_expr
2024-08-11 13:59:49 +02:00
gandeevanr
23a7d389b1
added unit tests for NewRowid
2024-08-08 19:04:10 -07:00
gandeevanr
a9cb8157b5
initial pass at implementing NewRowId
2024-08-07 09:04:09 -07:00
sonhmai
c104e50a66
core: add datetime modifiers helpers.
...
Building the helpers for integrating with supporting datetime Functions modifiers later.
2024-08-07 13:43:51 +07:00
Pekka Enberg
d4d7042373
Merge 'Use the correct integer PK column idx as the row-id alias' from GV
...
This pull request addresses [issue #256 ](https://github.com/penberg/limbo/issues/256 ).
However, it currently breaks insertions into tables that lack a row-id
alias due to the NewRowId not being implemented yet. I have created an
[issue](https://github.com/penberg/limbo/issues/275 ) to track this
problem and will submit a separate pull request to resolve it soon.
Closes #274
2024-08-06 08:11:10 +03:00
Pekka Enberg
5b7d112e74
Merge 'implementation of json function json(X)' from Jean Arhancet
...
Add the `json` function `json(X)` (related to the issue https://github.com/penberg/limbo/issues/127 )
Closes #230
2024-08-06 08:05:33 +03:00
JeanArhancet
15ab7955d3
add licences
2024-08-05 21:34:30 +02:00
gandeevanr
2b86f89d8d
use the correct integer PK column idx as the row-id alias
2024-08-04 18:53:54 -07:00
JeanArhancet
4050a3ebe2
refactor: impl json5 direclty
2024-08-04 12:47:08 +02:00
JeanArhancet
4d0a25c1b4
test: add json tcl test
2024-08-04 10:56:52 +02:00
JeanArhancet
8dbced6a39
refactor: delete Insn Json
2024-08-04 10:56:51 +02:00
JeanArhancet
552090cb29
feat: add json support
2024-08-04 10:54:27 +02:00
Pekka Enberg
31308f3699
core: Switch SQLite to use locking_mode EXCLUSIVE
...
Let's switch the SQLite benchmarks to use similar file locking protocol
as we do. Improves SQLite's performance by 2x, but Limbo is still
faster.
Before:
rusqlite/Prepare statement: 'SELECT * FROM users LIMIT 1'
time: [2.1027 µs 2.1239 µs 2.1563 µs]
thrpt: [463.75 Kelem/s 470.83 Kelem/s 475.57 Kelem/s]
After:
rusqlite/Execute prepared statement: 'SELECT * FROM users LIMIT 1'
time: [940.07 ns 944.16 ns 949.39 ns]
thrpt: [1.0533 Melem/s 1.0591 Melem/s 1.0638 Melem/s]
2024-08-03 21:13:50 +03:00
Pekka Enberg
89079d1ccd
Merge 'core: fix clippy' from Sonny
...
Closes #268
2024-08-03 17:53:51 +03:00
sonhmai
0e7bd95e4e
core: fix clippy
2024-08-03 20:14:26 +07:00
Pekka Enberg
a290b2f102
core: Open WAL file and parse header
2024-08-03 12:48:16 +03:00
