This PR fixes
[#1040](https://github.com/tursodatabase/limbo/issues/1040) and modifies
the `LIKE` function in the VDBE to work on expressions of all types like
SQLite.
Looking at how SQLite handles this, it gets the text value of the
expression regardless of its affinity. I used `exec_cast(exp, "TEXT")`
to achieve the same effect. Since most `LIKE` queries will probably be
done on `TEXT` expressions, I avoid casting the expression if it's
already `TEXT`. If either of the expressions was `NULL`, SQLite returns
nothing i.e. `NULL`. I also changed the unreachable arm message from
`Like on non-text registers` to `Like failed`.
The following queries produced the same results in Limbo:
```
SQLite version 3.46.1 2024-08-13 09:16:08 (UTF-16 console I/O)
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite> CREATE TABLE tbl (n NULL, i INTEGER, r REAL, t TEXT, b BLOB);
sqlite> INSERT INTO tbl VALUES(NULL,1,2.0,'a',X'0500');
sqlite> SELECT * FROM tbl;
|1|2.0|a|♣
sqlite> SELECT * FROM tbl WHERE n LIKE NULL;
sqlite> SELECT * FROM tbl WHERE n LIKE 'NULL';
sqlite> SELECT * FROM tbl WHERE n LIKE 1;
sqlite> SELECT * FROM tbl WHERE n LIKE 2.0;
sqlite> SELECT * FROM tbl WHERE n LIKE x'0500';
sqlite>
sqlite> SELECT * FROM tbl WHERE i LIKE NULL;
sqlite> SELECT * FROM tbl WHERE i LIKE 1;
|1|2.0|a|♣
sqlite> SELECT * FROM tbl WHERE i LIKE '1';
|1|2.0|a|♣
sqlite> SELECT * FROM tbl WHERE i LIKE 2.0;
sqlite> SELECT * FROM tbl WHERE i LIKE 1.0;
sqlite> SELECT * FROM tbl WHERE i LIKE x'0500';
sqlite>
sqlite> SELECT * FROM tbl WHERE r LIKE NULL;
sqlite> SELECT * FROM tbl WHERE r LIKE 2;
sqlite> SELECT * FROM tbl WHERE r LIKE 2.0;
|1|2.0|a|♣
sqlite> SELECT * FROM tbl WHERE r LIKE '2.0';
|1|2.0|a|♣
sqlite> SELECT * FROM tbl WHERE r LIKE 'a';
sqlite> SELECT * FROM tbl WHERE r LIKE x'0500';
sqlite>
sqlite> SELECT * FROM tbl WHERE t LIKE NULL;
sqlite> SELECT * FROM tbl WHERE t LIKE 1;
sqlite> SELECT * FROM tbl WHERE t LIKE 2.0;
sqlite> SELECT * FROM tbl WHERE t LIKE 'a';
|1|2.0|a|♣
sqlite> SELECT * FROM tbl WHERE t LIKE x'0500';
sqlite>
sqlite> SELECT * FROM tbl WHERE b LIKE NULL;
sqlite> SELECT * FROM tbl WHERE b LIKE 1;
sqlite> SELECT * FROM tbl WHERE b LIKE 2.0;
sqlite> SELECT * FROM tbl WHERE b LIKE 'a';
sqlite> SELECT * FROM tbl WHERE b LIKE x'0500';
|1|2.0|a|♣
sqlite> SELECT * FROM tbl WHERE b LIKE 'x''0500''';
sqlite> SELECT * FROM tbl WHERE b LIKE '♣';
sqlite>
sqlite> SELECT * FROM tbl WHERE 1 LIKE 1;
|1|2.0|a|♣
sqlite> SELECT * FROM tbl WHERE 2.0 LIKE 2.0;
|1|2.0|a|♣
sqlite> SELECT * FROM tbl WHERE 2.0 LIKE '2.0';
|1|2.0|a|♣
sqlite> SELECT * FROM tbl WHERE '2.0' LIKE 2.0;
|1|2.0|a|♣
sqlite> SELECT * FROM tbl WHERE '123.45' LIKE 123.45;
|1|2.0|a|♣
sqlite> SELECT * FROM tbl WHERE NULL LIKE NULL;
sqlite> SELECT * FROM tbl WHERE x'0500' LIKE x'0500';
|1|2.0|a|♣
sqlite> SELECT typeof(n), typeof(i), typeof(r), typeof(t), typeof(b) FROM tbl;
null|integer|real|text|blob
```
Though, these queries are very basic, and more testing could be done.
Closes#1044
Modified `cast_text_to_number` to be more compatible with SQLite. When
I was running some fuzz tests, I would eventually get errors due to
incorrect casting of text to `INTEGER` or `REAL`. Previously in code
there were 2 implementations of `cast_text_to_number`: one in
`core/vdbe/insn.rs` and one in `core/vdbe/mod.rs`. I consolidated the
casting to only one function. Previously, the `mod.rs` function was just
calling `checked_cast_text_to_numeric`, which was used in `MustBeInt`
opcode. Hopefully this fixes some of the CI testing issues we are
having. This was the query that prompted me to do this: `SELECT ( ( (
878352367 ) <> ( 29 ) ) ) = ( ( ( -4309097 ) / ( -37 || -149680985265412
) ) - 755066415 );`
Closes#1038
closes#977
In order to properly get #960 merged and keep some sort of the same API
we have now, we need to support URIs/query parameters for opening new
databases.
This PR doesn't attempt to implement anything useful with this, it only
handles parsing, but it will allow #960 to properly open a new file with
a specific VFS without having to entirely re-design the `open_file`
method/API. The existing option in that PR right now is less than ideal.
e.g. All of the existing methods already accept an `IO` impl
```rust
pub fn open_file(io: Arc<dyn IO>, path: &str) -> Result<Arc<Database>> {
// or
pub fn open(
io: Arc<dyn IO>,
page_io: Rc<dyn DatabaseStorage>,
wal: Rc<RefCell<dyn Wal>>,
shared_wal: Arc<RwLock<WalFileShared>>,
buffer_pool: Rc<BufferPool>,
) -> Result<Arc<Database>> {
```
Right now, most of the parsed query parameters are not options we
support yet, but I figured it's better to handle parsing them now and
using them later on when we support them.
Also, if this looks way overly complicated for what it does... that's
because the cross platform edge-cases are a super pain in the ass.
Closes#1039
This is a first step to supporting [sqlite column
affinity](https://www.sqlite.org/datatype3.html) properly in limbo --
just adds an `affinity()` function to `Column` that can then be used
elsewhere
Closes#1030
We were not evaluating constant conditions (e.g '1 IS NULL')
when there were no tables referenced in the query, because
our WHERE term evaluation was based on "during which loop"
to evaluate them. However, when there are no tables, there are
no loops, so they were never evaluated.
Fix codegen for binary functions and add fuzz test for math functions
(we need to compile `rusqlite` with `-DSQLITE_ENABLE_MATH_FUNCTIONS` in
order to bundle sqlite with math functions compiled)
Reviewed-by: Jussi Saurio (@jussisaurio)
Closes#1015
Align `substr` implementation with SQLite spec
(https://www.sqlite.org/lang_corefunc.html#substr):
> The substr(X,Y,Z) function returns a substring of input string X that
begins with the Y-th character and which is Z characters long. If Z is
omitted then substr(X,Y) returns all characters through the end of the
string X beginning with the Y-th. The left-most character of X is number
1. If Y is negative then the first character of the substring is found
by counting from the right rather than the left. If Z is negative then
the abs(Z) characters preceding the Y-th character are returned. If X is
a string then characters indices refer to actual UTF-8 characters. If X
is a BLOB then the indices refer to bytes.
Reviewed-by: Jussi Saurio (@jussisaurio)
Closes#1013
According to SQLite documentation, the way to use these instructions
is to compare the seek key to the index key as you would with the
Compare opcode. The compare opcode states:
"Compare two vectors of registers in reg(P1)..reg(P1+P3-1)
(call this vector "A") and in reg(P2)..reg(P2+P3-1) ("B")."
In other words, we should compare the same number of columns from each,
not compare the entire keys.
This fixes a few Clickbench queries returning incorrect results, and
so closes#1009
---
Future work: support index seek keys that use multiple columns. Our
index seek is many times slower than SQLite because we're not utilizing
all the possible columns -- instead we just use the first index column
to seek.
I added the two opcodes as an initial step. They are pretty easy to
implement since we already have the counterparts i.e., IdxGE and IdxGT
Is there a design reason behind their omission @penberg @PThorpe92?
I noticed the same for SeekLE and SeekLT.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#1010
easy implementation, sqlite claims it is a noop now
"This pragma no longer functions. It has become a no-op. The
capabilities formerly provided by PRAGMA legacy_file_format are now
available using the SQLITE_DBCONFIG_LEGACY_FILE_FORMAT option to the
sqlite3_db_config() C-language interface."
Closes#1007
