First attempt at closing #1212. Also with this PR, I added the option of
using `with` syntax for `TestLimboShell`. It automatically closes the
shell on error, and facilitates error handling overall. If this is
merged, I can update the other python tests to use `with` as well.
Reviewed-by: Preston Thorpe (@PThorpe92)
Closes#1230
Some more helper utilities extracted out from #1351 to make the surface
area of that PR smaller
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#1353
Apply affinities to a range of P2 registers starting with P1.
P4 is a string that is P2 characters long. The N-th character of the string indicates the column affinity that should be used for the N-th memory cell in the range.
If P4==0 then register P3 holds a blob constructed by MakeRecord. If P4>0 then register P3 is the first of P4 registers that form an unpacked record.
Cursor P1 is on an index btree. If the record identified by P3 and P4 is not the prefix of any entry in P1 then a jump is made to P2. If P1 does contain an entry whose prefix matches the P3/P4 record then control falls through to the next instruction and P1 is left pointing at the matching entry.
This operation leaves the cursor in a state where it cannot be advanced in either direction. In other words, the Next and Prev opcodes do not work after this operation.
When the python tests fail, they will sometimes truncate the output if
it is smaller than the `PIPE_BUFF` size. With this fix we can now
properly print the backtrace, when the program panics.
# Before
This is the problematic CI output from #1331 that led me to fix this.
In this case, it was already truncating the output of the `assert`
prints.
```
./testing/cli_tests/extensions.py
Extension ./target/debug/liblimbo_regexp loaded successfully.
Testing: uuid functions are registered properly with ext loaded
Testing: scalar alias's are registered properly
Testing: median agg function returns null when ext not loaded
Testing: median agg function works
Testing: median agg function works with odd number of elements
Testing: test aggregate percentile function with 2 arguments works
Testing: test aggregate percentile function with 1 argument works
Testing: crypto_blake3 returns null when ext not loaded
Testing: blake3 should encrypt correctly
Testing: md5 should encrypt correctly
Testing: sha1 should encrypt correctly
Testing: sha256 should encrypt correctly
Testing: sha384 should encrypt correctly
Testing: sha512 should encrypt correctly
Testing: base32 should encode correctly
Testing: base32 should decode correctly
Testing: base64 should encode correctly
Testing: base64 should decode correctly
Testing: base85 should encode correctly
Testing: base85 should decode correctly
Testing: hex should encode correctly
Testing: hex should decode correctly
Testing: url should encode correctly
Testing: url should decode correctly
Testing: ipfamily function returns null when ext not loaded
Testing: ipfamily function returns 4 for IPv4
Testing: ipfamily function returns 6 for IPv6
Testing: ipcontains function returns 1 for IPv4
Testing: ipcontains function returns 0 for IPv4
Testing: iphost function returns the host for IPv4
Testing: iphost function returns the host for IPv6
Testing: ipmasklen function returns the mask length for IPv4
Testing: ipmasklen function returns the mask length for IPv6
Testing: ipnetwork function returns the flattened CIDR for IPv4
Testing: ipnetwork function returns the network for IPv6
Testing: testvfs not loaded
Testing: testvfs extension loaded
thread 'main' panicked at core/storage/pager.rs:61:38:
called `Option::unwrap()` on a `None` value
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Test FAILED: Error encountered in Limbo shell.
make: *** [Makefile:70: test-extensions] Error 1
```
# After
```
with-env {RUST_BACKTRACE:1} {make test-extensions}
cargo build
Compiling limbo_regexp v0.0.19-pre.4 (/Users/pedro/Projects/limbo/extensions/regexp)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.51s
cargo build --package limbo_regexp
Compiling limbo_regexp v0.0.19-pre.4 (/Users/pedro/Projects/limbo/extensions/regexp)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.19s
./testing/cli_tests/extensions.py
Extension ./target/debug/liblimbo_regexp loaded successfully.
Testing: uuid functions are registered properly with ext loaded
Testing: scalar alias's are registered properly
Testing: median agg function returns null when ext not loaded
Testing: median agg function works
Testing: median agg function works with odd number of elements
Testing: test aggregate percentile function with 2 arguments works
Testing: test aggregate percentile function with 1 argument works
Testing: crypto_blake3 returns null when ext not loaded
Testing: blake3 should encrypt correctly
Testing: md5 should encrypt correctly
Testing: sha1 should encrypt correctly
Testing: sha256 should encrypt correctly
Testing: sha384 should encrypt correctly
Testing: sha512 should encrypt correctly
Testing: base32 should encode correctly
Testing: base32 should decode correctly
Testing: base64 should encode correctly
Testing: base64 should decode correctly
Testing: base85 should encode correctly
Testing: base85 should decode correctly
Testing: hex should encode correctly
Testing: hex should decode correctly
Testing: url should encode correctly
Testing: url should decode correctly
Testing: ipfamily function returns null when ext not loaded
Testing: ipfamily function returns 4 for IPv4
Testing: ipfamily function returns 6 for IPv6
Testing: ipcontains function returns 1 for IPv4
Testing: ipcontains function returns 0 for IPv4
Testing: iphost function returns the host for IPv4
Testing: iphost function returns the host for IPv6
Testing: ipmasklen function returns the mask length for IPv4
Testing: ipmasklen function returns the mask length for IPv6
Testing: ipnetwork function returns the flattened CIDR for IPv4
Testing: ipnetwork function returns the network for IPv6
Testing: testvfs not loaded
Testing: testvfs extension loaded
thread 'main' panicked at core/storage/pager.rs:61:38:
called `Option::unwrap()` on a `None` value
stack backtrace:
0: rust_begin_unwind
at /rustc/90b35a6239c3d8bdabc530a6a0816f7ff89a0aaf/library/std/src/panicking.rs:665:5
1: core::panicking::panic_fmt
at /rustc/90b35a6239c3d8bdabc530a6a0816f7ff89a0aaf/library/core/src/panicking.rs:74:14
2: core::panicking::panic
at /rustc/90b35a6239c3d8bdabc530a6a0816f7ff89a0aaf/library/core/src/panicking.rs:148:5
3: core::option::unwrap_failed
at /rustc/90b35a6239c3d8bdabc530a6a0816f7ff89a0aaf/library/core/src/option.rs:2012:5
4: core::option::Option<T>::unwrap
at /Users/pedro/.rustup/toolchains/1.83.0-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/option.rs:972:21
5: limbo_core::storage:📟:Page::get_contents
at ./core/storage/pager.rs:61:9
6: limbo_core::storage::btree::BTreeCursor::balance_non_root
at ./core/storage/btree.rs:1723:40
7: limbo_core::storage::btree::BTreeCursor::balance
at ./core/storage/btree.rs:1570:35
8: limbo_core::storage::btree::BTreeCursor::insert_into_page
at ./core/storage/btree.rs:1512:35
9: limbo_core::storage::btree::BTreeCursor::insert
at ./core/storage/btree.rs:3024:31
10: limbo_core::vdbe::execute::op_insert
at ./core/vdbe/execute.rs:3654:23
11: limbo_core::vdbe::Program::step
at ./core/vdbe/mod.rs:379:23
12: limbo_core::Statement::step
at ./core/lib.rs:582:9
13: limbo::app::Limbo::print_query_result
at ./cli/app.rs:657:27
14: limbo::app::Limbo::run_query
at ./cli/app.rs:420:20
15: limbo::app::Limbo::handle_input_line
at ./cli/app.rs:527:13
16: limbo::main
at ./cli/main.rs:29:31
17: core::ops::function::FnOnce::call_once
at /Users/pedro/.rustup/toolchains/1.83.0-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
Testing: Tested large write to testfs
Test FAILED: Test failed
SQL: SELECT count(*) FROM test;
Actual:
None
make: *** [test-extensions] Error 1
```
Closes#1346
Sqlite reference implementation: https://github.com/sqlite/sqlite/blob/8
37dc09bce7de8971c7488b70cf5da93c60fbed0/src/vdbe.c#L2558
We did not support blobs before and our ordering of the match statements
were incorrect when one of the arguments was NULL
Closes#1337
Currently, to run the fuzz test you need to remove an assert statement.
EDIT: @diegoreis42 pointed it out to me that the method had just changed
actually. Made the change to use `row.len()` instead
# Problem
```
cargo +nightly fuzz run --target=aarch64-apple-darwin expression
Compiling limbo_core v0.0.19-pre.4 (/Users/pedro/Projects/limbo/core)
Compiling limbo-fuzz v0.0.0 (/Users/pedro/Projects/limbo/fuzz)
error[E0599]: `&limbo_core::vdbe::Row` is not an iterator
--> fuzz_targets/expression.rs:198:36
|
198 | assert_eq!(row.count(), 1, "expr: {:?}", expr);
| ^^^^^
| |
| `&limbo_core::vdbe::Row` is not an iterator
| private field, not a method
|
::: /Users/pedro/Projects/limbo/core/vdbe/mod.rs:221:1
|
221 | pub struct Row {
| -------------- doesn't satisfy `limbo_core::vdbe::Row: std::iter::Iterator`
|
= note: the following trait bounds were not satisfied:
`&limbo_core::vdbe::Row: std::iter::Iterator`
which is required by `&mut &limbo_core::vdbe::Row: std::iter::Iterator`
`limbo_core::vdbe::Row: std::iter::Iterator`
which is required by `&mut limbo_core::vdbe::Row: std::iter::Iterator`
For more information about this error, try `rustc --explain E0599`.
error: could not compile `limbo-fuzz` (bin "expression") due to 1 previous error
Error: failed to build fuzz script: ASAN_OPTIONS="detect_odr_violation=0" RUSTFLAGS="-Cpasses=sancov-module -Cllvm-args=-sanitizer-coverage-level=4 -Cllvm-args=-sanitizer-coverage-inline-8bit-counters -Cllvm-args=-sanitizer-coverage-pc-table -Cllvm-args=-sanitizer-coverage-trace-compares --cfg fuzzing -Clink-dead-code -Zsanitizer=address -Cdebug-assertions -C codegen-units=1" "cargo" "build" "--manifest-path" "/Users/pedro/Projects/limbo/fuzz/Cargo.toml" "--target" "aarch64-apple-darwin" "--release" "--config" "profile.release.debug=true" "--bin" "expression"
```
Closes#1334
Fixes#1298
- Fixes Limbo trying to use an index using a WHERE clause constraint
that refers to the same table on both sides, e.g. `WHERE t.x = t.x`
- Fixes not using indexes when the relevant expression is paren wrapped,
e.g.
- `SELECT * FROM t WHERE (indexcol) > 5`
- `SELECT * FROM t WHERE (indexcol > 5)`
- Changes existing table logical expr fuzz test to have primary keys
(which helped me find both issues above)
Closes#1300
Fixed is_empty assertion in #1203 , but simulator still has error.
```sh
[ERROR] error Internal error: select '(engrossing_berger < X'6566651A3C70278D4E200657551D8071A1' AND competitive_petit > 1236742147.9451914)' should return no values for table 'super_becky'
```
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#1308
