This patch brings a bunch of quality of life improvements to encryption:
1. Previously, we just let any string to be used as a key. I have
updated the `PRAGMA hexkey=''` to get the key in hex. I have also
renamed from `key`, because that will be used to get passphrase
2. Added `PRAGMA cipher` so that now users can select which cipher they
want to use (for now, either `aegis256` or `aes256gcm`)
3. We now set the encryption context when both cipher and key are set
I also updated tests to reflect this.
Reviewed-by: Preston Thorpe <preston@turso.tech>
Closes#2779
My goal with this patch is to be able to implement the ProjectOperator
for DBSP circuits using VDBE for expression evaluation.
*not* doing so is dangerous for the following reason: we will end up
with different, subtle, and incompatible behavior between SQLite
expressions if they are used in views versus outside of views.
In fact, even in our prototype had them: our projection tests, which
used to pass, were actually wrong =) (sqlite would return something
different if those functions were executed outside the view context)
For optimization reasons, we single out trivial expressions: they don't
have go through VDBE. Trivial expressions are expressions that only
involve Columns, Literals, and simple operators on elements of the same
type. Even type coercion takes this out of the realm of trivial.
Everything that is not trivial, is then translated with translate_expr -
in the same way SQLite will, and then compiled with VDBE.
We can, over time, make this process much better. There are essentially
infinite opportunities for optimization here. But for now, the main
warts are:
* VDBE execution needs a connection
* There is no good way in VDBE to pass parameters to a program.
* It is almost trivial to pollute the original connection. For example,
we need to issue HALT for the program to stop, but seeing that halt
will usually cause the program to try and halt the original program.
Subprograms, like the ones we use in triggers are a possible solution,
but they are much more expensive to execute, especially given that our
execution would essentially have to have a program with no other role
than to wrap the subprogram.
Therefore, what I am doing is:
* There is an in-memory database inside the projection operator (an
obvious optimization is to share it with *all* projection operators).
* We obtain a connection to that database when the operator is created
* We use that connection to execute our VDBE, which offers a clean, safe
and isolated way to execute the expression.
* We feed the values to the program manually by editing the registers
directly.
To be used in DBSP-based projections. This will compile an expression
to VDBE bytecode and execute it.
To do that we need to add a new type of Expression, which we call a
Register.
This is a way for us to pass parameters to a DBSP program which will be
not columns or literals, but inputs from the DBSP deltas.
Added `from_hex_string` which gets us `EncryptionKey` from a
hex string. Now we can use securely generated keys, like from openssl
$ openssl rand -hex 32
Previously, the encryption module had hardcoded a lot of things. This
refactor makes it slightly nice and makes it configurable.
Right now cipher algorithm is assumed and hardcoded, I will make that
configurable in the upcoming PR
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#2722
SQLite does not store the rowid alias column in the record at all
when it is a rowid alias, because the rowid is always stored anyway
in the record header.
Found when running simulator in #2641
All indexes store the rowid as the last column, so whenever the rowid of
a given row changes the index entry must also be deleted and reinserted
with the new index.
Reviewed-by: Nikita Sivukhin (@sivukhin)
Closes#2712
UPDATE should skip over the UNIQUE constraint failure if the existing
row it found during the check has the same rowid as the row we are
currently updating
Same deal as #2700, except this time in UPDATE. Nothing tests this on
`main` so not caught.
I will later put #2641 into mergeable condition so it will catch all of
these going forward.
Reviewed-by: Nikita Sivukhin (@sivukhin)
Closes#2710
We were not generating table_info for views. This PR fixes it. We were
so far storing columns as strings with just their names - since this is
all we needed - but we will move now to store Columns. We need to
convert the names to Column anyway for table_info to work.
Closes#2625
We were not generating table_info for views. This PR fixes it. We were
so far storing columns as strings with just their names - since this is
all we needed - but we will move now to store Columns. We need to
convert the names to Column anyway for table_info to work.
We had a bug where we were checking for duplicates in the DISTINCT
index based on both the result column count plus any ORDER BY columns
not present in the DISTINCT clause.
This is wrong, so fix it by only using the result columns for the
dedupe check.
In case an ORDER BY column exactly matches a result column in the SELECT,
the insertion of the result column into the ORDER BY sorter can be skipped
because it's already necessarily inserted as a sorting column.
For this reason we have a mapping to know what index a given result column
has in the order by sorter.
This commit makes that mapping much simpler.
Implements the unlikely(X) function. Removes runtime implementations of
likely(), unlikely() and likelihood(), replacing them with panics if
they reach the VDBE.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#2559
A lot of the structures we have - like the ones under Schema, are
specific for materialized views. In preparation to adding normal views,
rename them, so things are less confusing.
