Closes#2946
currently we always evaluate the binary expression, then coerce it to
zero/null with the `ZeroOrNull` instruction, and then emit a separate
jump.
this is fine for non-conditional expressions where we are using the
value itself (e.g. in a SELECT result column), but in conditionals we
don't care about that at all and just want to either jump or not jump.
so: try to keep the spirit of code reuse, but still have distinct
implementations for conditionals and non-conditionals.
Reviewed-by: Preston Thorpe <preston@turso.tech>
Closes#2947
Attempt to improve performance of `SELECT COUNT(*) FROM t WHERE 1 = 1`
(redundant filter added to force sqlite and tursodb to iterate over rows
instead of applying count optimization).
Before that PR turso with hot cache runs for ~440ms.
After that PR turso with hot cache runs for ~200ms.
This PR eliminates small things on a hot path and also introduce
separate flow for `get_next_record` which just increment cell idx if its
safe to do that - without much additional overhead.
Also, this PR eliminates RefCell/Cell from PageStack because there were
completely unnecessary.
Reviewed-by: Preston Thorpe <preston@turso.tech>
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#2887
This adds the [`json_each`](https://sqlite.org/json1.html#the_json_each_
and_json_tree_table_valued_functions) TVF. Only the 1-arg version is
supported for now.
As suggested in the comments on this PR, I've also extended the virtual
table system to support internal TVF's, as opposed to extensions and
pragma TVF's.
Reviewed-by: Preston Thorpe <preston@turso.tech>
Closes#2691
currently we always evaluate the binary expression, then coerce it
to zero/null with the `ZeroOrNull` instruction, and then emit a separate
jump.
this is fine for non-conditional expressions where we are using the value
itself (e.g. in a SELECT result column), but in conditionals we don't care
about that at all and just want to jump.
so: try to keep the spirit of code reuse, but still have distinct implementations
for conditionals and non-conditionals.
This PR replaces the existing LRU cache implementation with the sieve
algo, providing better performance and simpler code.
Same intrusive linked-list without pointer chasing and no per-
page/insert heap allocations.
Benchmarks look really good so far, although some did show some
regression (I think it's more likely that they were flakey and its not
representative on either end, but I do believe this to be considerably
more performant)
Best numbers I have seen yet on Mobibench locally, with 1473 tx/s.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Reviewed-by: Nikita Sivukhin (@sivukhin)
Closes#2885
On the syscall IO backend, on TPC-H query 12, the _dominating_ part of
the stack trace is trying to construct affinities from a character,
failing, allocating an error&string, and then immediately falling back
to Blob affinity and dropping the error&string.
Since I'm on vacation I won't spend cycles on figuring out why we are
passing an incorrect affinity in `flags.get_affinity()` and instead make
this lazy PR just to improve performance and stop doing silly things :]
Reviewed-by: Preston Thorpe <preston@turso.tech>
Closes#2941
On the syscall IO backend, on TPC-H query 12, the _dominating_ part
of the stack trace is trying to construct affinities from a character,
failing, allocating an error&string, and then immediately falling back to
Blob affinity and dropping the error&string.
Since I'm on vacation I won't spend cycles on figuring out why we are passing
an incorrect affinity in `flags.get_affinity()` and instead make this lazy PR
just to improve performance and stop doing silly things :]
Add handling malformed inputs to function `read_varint` and test cases.
```
# 9 byte truncated to 8
read_varint(&[0x81, 0x80, 0x80, 0x80, 0x80, 0x80, 0x80, 0x80])
before -> panic index out of bounds: the len is 8 but the index is 8
after -> LimboError
# bits set without end
read_varint(&[0x80; 9])
before -> Ok((128, 9))
after -> LimboError
```
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#2904
fix#2937 , we dont need to check comment in cli because parser already
handled it (i guest).
How to reproduce:
```sh
turso> --whatCREATE TABLE users (
id INT PRIMARY KEY,
first_name VARCHAR(50),
age INT
);
```
Closes#2938
This fairly long commit implements persistence for materialized view. It
is hard to split because of all the interdependencies between
components, so it is a one big thing. This commit message will at least
try to go into details about the basic architecture.
Materialized Views as tables
============================
Materialized views are now a normal table - whereas before they were a
virtual table. By making a materialized view a table, we can reuse all
the infrastructure for dealing with tables (cursors, etc).
One of the advantages of doing this is that we can create indexes on
view columns. Later, we should also be able to write those views to
separate files with ATTACH write.
Materialized Views as Zsets
===========================
The contents of the table are a ZSet: rowid, values, weight. Readers
will notice that because of this, the usage of the ZSet data structure
dwindles throughout the codebase. The main difference between our
materialized ZSet and the standard DBSP ZSet, is that obviously ours is
backed by a BTree, not a Hash (since SQLite tables are BTrees)
Aggregator State
================
In DBSP, the aggregator nodes also have state. To store that state,
there is a second table. The table holds all aggregators in the view,
and there is one table per view. That is
__turso_internal_dbsp_state_{view_name}. The format of that table is
similar to a ZSet: rowid, serialized_values, weight. We serialize the
values because there will be many aggregators in the table. We can't
rely on a particular format for the values.
The Materialized View Cursor
============================
Reading from a Materialized View essentially means reading from the
persisted ZSet, and enhancing that with data that exists within the
transaction. Transaction data is ephemeral, so we do not materialize
this anywhere: we have a carefully crafted implementation of seek that
takes care of merging weights and stitching the two sets together.
Closes#2921
