EDIT:
Ok this should finally be all set. There were some massive changes that
had taken place since the original PR 🫠🫠
merge conflicts were serious
As soon as this is merged, I will finish the rest of what is needed for
`CREATE VIRTUAL TABLE some_table USING some_extension('some argument or
sql');`
I added tests for the series extension.
This PR is already big and covers a LOT of surface area, so we should
ideally try to get this in first, and I'll go back and complete the rest
of the functionality 👍Closes#858
Move result row to `ProgramState` to mimic what SQLite does where `Vdbe`
struct has a `pResultRow` member. This makes it easier to deal with
result lifetime, but more importantly, eventually lazily parse values at
the edges of the API.
Closes#904
Move result row to `ProgramState` to mimic what SQLite does where `Vdbe`
struct has a `pResultRow` member. This makes it easier to deal with result
lifetime, but more importantly, eventually lazily parse values at the edges of
the API.
To make implementation of DiskANN in limbo easier, I'm moving `vector`
from `extensions` to core.
Now `vector` related function are exposed via `Function` op code.
I've defined a new enum called `VectorFunc` to group the vector related
functions.
The `vector.test` TCL test runs fine.
```sql
limbo> SELECT vector_extract(vector('[]'));
[]
limbo> SELECT vector_extract(vector(' [ 1 , 2 , 3 ] '));
[1,2,3]
limbo> SELECT vector_extract(vector('[-1000000000000000000]'));
[-1000000000000000000]
limbo> SELECT vector_distance_cos(vector('[1,2,3]'), vector('[3,2,1]'));
0.2857142686843872
```
Closes#902
Use knowledge of query plan to inform how much memory to initially
allocate for `ProgramBuilder` vectors
Some of them are exact, some are semi-random estimates
```sql
Prepare `SELECT 1`/Limbo/SELECT 1
time: [756.93 ns 758.11 ns 759.59 ns]
change: [-4.5974% -4.3153% -4.0393%] (p = 0.00 < 0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
2 (2.00%) low severe
1 (1.00%) low mild
3 (3.00%) high mild
1 (1.00%) high severe
Prepare `SELECT * FROM users LIMIT 1`/Limbo/SELECT * FROM users LIMIT 1
time: [1.4739 µs 1.4769 µs 1.4800 µs]
change: [-7.9364% -7.7171% -7.4979%] (p = 0.00 < 0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou...`
time: [3.7440 µs 3.7520 µs 3.7596 µs]
change: [-5.4627% -5.1578% -4.8445%] (p = 0.00 < 0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe
```
Closes#899
- Remove eagerly allocated `name` from `ResultSetColumn`
- `ResultSetColumn` can calculate `name()` on demand:
- if it has an alias (`foo as bar`), use that
- if it is a column reference, use that
- otherwise return none, and callers can assign it a placeholder
name (like `column_1`)
- move the `plan.result_columns` and `plan.table_references` to
`Program` after preparing statement is done, so that column names can be
returned upon request
- make `name` in `Column` optional, not needed for pseudo tables and
sorters so avoids an extra string allocation
```sql
Prepare `SELECT 1`/Limbo/SELECT 1
time: [756.80 ns 758.27 ns 760.04 ns]
change: [-3.3257% -3.0252% -2.7035%] (p = 0.00 < 0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
2 (2.00%) low severe
3 (3.00%) low mild
1 (1.00%) high mild
2 (2.00%) high severe
Prepare `SELECT * FROM users LIMIT 1`/Limbo/SELECT * FROM users LIMIT 1
time: [1.4646 µs 1.4669 µs 1.4696 µs]
change: [-6.4769% -6.2021% -5.9137%] (p = 0.00 < 0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
1 (1.00%) low severe
3 (3.00%) low mild
3 (3.00%) high severe
Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou...`
time: [3.7256 µs 3.7311 µs 3.7376 µs]
change: [-4.5195% -4.2192% -3.9309%] (p = 0.00 < 0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
1 (1.00%) low severe
2 (2.00%) low mild
2 (2.00%) high mild
```
Closes#898
This PR adds support for `json_set`.
There are three helper functions added:
1. `json_path_from_owned_value`, this function turns an `OwnedValue`
into a `JsonPath`.
2. `find_or_create_target`, this function is similar to `find_target`
with the added bonus of creating the target if it doesn't exist. There
is a caveat with this function and that is that it will create
objects/arrays as it goes, meaning if you send `{}` into it and try
getting the path `$.some.nested.array[123].field`, it will return
`{"some":{"nested":array:[]}}` since creation of `some`, `nested` and
`array` will succeed, but accessing element `123` will fail.
3. `create_and_mutate_json_by_path`, this function is very similar to
`mutate_json_by_path` but calls `find_or_create_target` instead of
`find_target`
Related to #127Closes#878
This pulls the MVCC implementation based on Hekaton me, @psarna, and
@avinassh did two years ago. No need to get excited, it's not integrated
to the Limbo core, just adds bunch of code likely full of bugs. Next
step is to see how to integrate this under a feature flag to query
executor and wire up transaction log to emit WAL.
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#896
