The initial commits fix issues and plug gaps between ungrouped and
grouped aggregations.
The final commit consolidates the code that emits `AggStep` to prevent
future disparities between the two.
Reviewed-by: Preston Thorpe <preston@turso.tech>
Closes#2867
This bug occurred when arguments were read for the GROUP BY sorter — all
arguments were incorrectly resolved to the first column. Added tests
confirm that aggregates now work correctly both with and without the
sorter.
This commit consolidates the creation of the Aggregate struct, which was
previously handled differently in `prepare_one_select_plan` and
`resolve_aggregates`. That discrepancy caused inconsistent handling of
zero-argument aggregates.
The queries added in the new tests would previously trigger a panic.
Instead of using static elements, use a dynamically generated DBSP-
circuit to keep views.
The DBSP circuit is generated from the logical plan, which only supports
enough for us to generate the DBSP circuit at the moment.
The state of the view is still kept inside the IncrementalView, instead
of materialized at the operator level. As a consequence, this still
depends on us always populating the view at startup. Fixing this is the
next step.
Closes#2815
Instead of making a new test that would be the same, just updated this one to show that sqlite and turso (with this pr) match and isnt case sensitive in strict tables.
My goal with this patch is to be able to implement the ProjectOperator
for DBSP circuits using VDBE for expression evaluation.
*not* doing so is dangerous for the following reason: we will end up
with different, subtle, and incompatible behavior between SQLite
expressions if they are used in views versus outside of views.
In fact, even in our prototype had them: our projection tests, which
used to pass, were actually wrong =) (sqlite would return something
different if those functions were executed outside the view context)
For optimization reasons, we single out trivial expressions: they don't
have go through VDBE. Trivial expressions are expressions that only
involve Columns, Literals, and simple operators on elements of the same
type. Even type coercion takes this out of the realm of trivial.
Everything that is not trivial, is then translated with translate_expr -
in the same way SQLite will, and then compiled with VDBE.
We can, over time, make this process much better. There are essentially
infinite opportunities for optimization here. But for now, the main
warts are:
* VDBE execution needs a connection
* There is no good way in VDBE to pass parameters to a program.
* It is almost trivial to pollute the original connection. For example,
we need to issue HALT for the program to stop, but seeing that halt
will usually cause the program to try and halt the original program.
Subprograms, like the ones we use in triggers are a possible solution,
but they are much more expensive to execute, especially given that our
execution would essentially have to have a program with no other role
than to wrap the subprogram.
Therefore, what I am doing is:
* There is an in-memory database inside the projection operator (an
obvious optimization is to share it with *all* projection operators).
* We obtain a connection to that database when the operator is created
* We use that connection to execute our VDBE, which offers a clean, safe
and isolated way to execute the expression.
* We feed the values to the program manually by editing the registers
directly.
There is a distinction between tests that verify extension-specific
behavior and those that verify interactions between the database engine
and extensions. Previously, both types of tests were kept in
`extensions.py`. With this new framework, we can extract the latter type
of tests from `extensions.py` into TCL. This cleans up `extensions.py`
and enables compatibility testing with SQLite at no extra cost.
I’m currently working on supporting outer joins involving TVFs and
planning to add more tests that exercise the database’s handling of
virtual tables, so I decided to do this refactoring first.
In the future, we may consider moving extension-specific tests to TCL as
well, especially those that have counterparts in SQLite or sqlean.
Reviewed-by: Preston Thorpe <preston@turso.tech>
Closes#2556
These tests verify interactions between the database engine and TVFs.
They happen to use generate_series, but they are not intended to test
the behavior of any specific extension. Tests that verify generate_series
specific behavior remain in extensions.py.
There is a distinction between tests that verify extension-specific
behavior and tests that verify interactions between the database engine
and extensions. Previously, both types of tests were kept in extensions.py.
With this new framework, we can extract the latter type of tests from
extensions.py into TCL. This cleans up extensions.py and provides
compatibility testing with SQLite at no extra cost.
To demonstrate the framework’s usage, tests verifying the handling of
virtual tables were extracted to TCL.
In the future, we may consider moving extension-specific tests to TCL as
well, especially those that have counterparts in SQLite or sqlean.
We had a bug where we were checking for duplicates in the DISTINCT
index based on both the result column count plus any ORDER BY columns
not present in the DISTINCT clause.
This is wrong, so fix it by only using the result columns for the
dedupe check.
Now that we actually implemented the statement parsing around views,
implementing normal SQLite views is relatively trivial, as they are just
an alias to a query.
We'll implement them now to get them out of the way, and then I'll go
back to DBSP
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#2591
