Some more helper utilities extracted out from #1351 to make the surface
area of that PR smaller
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#1353
When the python tests fail, they will sometimes truncate the output if
it is smaller than the `PIPE_BUFF` size. With this fix we can now
properly print the backtrace, when the program panics.
# Before
This is the problematic CI output from #1331 that led me to fix this.
In this case, it was already truncating the output of the `assert`
prints.
```
./testing/cli_tests/extensions.py
Extension ./target/debug/liblimbo_regexp loaded successfully.
Testing: uuid functions are registered properly with ext loaded
Testing: scalar alias's are registered properly
Testing: median agg function returns null when ext not loaded
Testing: median agg function works
Testing: median agg function works with odd number of elements
Testing: test aggregate percentile function with 2 arguments works
Testing: test aggregate percentile function with 1 argument works
Testing: crypto_blake3 returns null when ext not loaded
Testing: blake3 should encrypt correctly
Testing: md5 should encrypt correctly
Testing: sha1 should encrypt correctly
Testing: sha256 should encrypt correctly
Testing: sha384 should encrypt correctly
Testing: sha512 should encrypt correctly
Testing: base32 should encode correctly
Testing: base32 should decode correctly
Testing: base64 should encode correctly
Testing: base64 should decode correctly
Testing: base85 should encode correctly
Testing: base85 should decode correctly
Testing: hex should encode correctly
Testing: hex should decode correctly
Testing: url should encode correctly
Testing: url should decode correctly
Testing: ipfamily function returns null when ext not loaded
Testing: ipfamily function returns 4 for IPv4
Testing: ipfamily function returns 6 for IPv6
Testing: ipcontains function returns 1 for IPv4
Testing: ipcontains function returns 0 for IPv4
Testing: iphost function returns the host for IPv4
Testing: iphost function returns the host for IPv6
Testing: ipmasklen function returns the mask length for IPv4
Testing: ipmasklen function returns the mask length for IPv6
Testing: ipnetwork function returns the flattened CIDR for IPv4
Testing: ipnetwork function returns the network for IPv6
Testing: testvfs not loaded
Testing: testvfs extension loaded
thread 'main' panicked at core/storage/pager.rs:61:38:
called `Option::unwrap()` on a `None` value
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Test FAILED: Error encountered in Limbo shell.
make: *** [Makefile:70: test-extensions] Error 1
```
# After
```
with-env {RUST_BACKTRACE:1} {make test-extensions}
cargo build
Compiling limbo_regexp v0.0.19-pre.4 (/Users/pedro/Projects/limbo/extensions/regexp)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.51s
cargo build --package limbo_regexp
Compiling limbo_regexp v0.0.19-pre.4 (/Users/pedro/Projects/limbo/extensions/regexp)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.19s
./testing/cli_tests/extensions.py
Extension ./target/debug/liblimbo_regexp loaded successfully.
Testing: uuid functions are registered properly with ext loaded
Testing: scalar alias's are registered properly
Testing: median agg function returns null when ext not loaded
Testing: median agg function works
Testing: median agg function works with odd number of elements
Testing: test aggregate percentile function with 2 arguments works
Testing: test aggregate percentile function with 1 argument works
Testing: crypto_blake3 returns null when ext not loaded
Testing: blake3 should encrypt correctly
Testing: md5 should encrypt correctly
Testing: sha1 should encrypt correctly
Testing: sha256 should encrypt correctly
Testing: sha384 should encrypt correctly
Testing: sha512 should encrypt correctly
Testing: base32 should encode correctly
Testing: base32 should decode correctly
Testing: base64 should encode correctly
Testing: base64 should decode correctly
Testing: base85 should encode correctly
Testing: base85 should decode correctly
Testing: hex should encode correctly
Testing: hex should decode correctly
Testing: url should encode correctly
Testing: url should decode correctly
Testing: ipfamily function returns null when ext not loaded
Testing: ipfamily function returns 4 for IPv4
Testing: ipfamily function returns 6 for IPv6
Testing: ipcontains function returns 1 for IPv4
Testing: ipcontains function returns 0 for IPv4
Testing: iphost function returns the host for IPv4
Testing: iphost function returns the host for IPv6
Testing: ipmasklen function returns the mask length for IPv4
Testing: ipmasklen function returns the mask length for IPv6
Testing: ipnetwork function returns the flattened CIDR for IPv4
Testing: ipnetwork function returns the network for IPv6
Testing: testvfs not loaded
Testing: testvfs extension loaded
thread 'main' panicked at core/storage/pager.rs:61:38:
called `Option::unwrap()` on a `None` value
stack backtrace:
0: rust_begin_unwind
at /rustc/90b35a6239c3d8bdabc530a6a0816f7ff89a0aaf/library/std/src/panicking.rs:665:5
1: core::panicking::panic_fmt
at /rustc/90b35a6239c3d8bdabc530a6a0816f7ff89a0aaf/library/core/src/panicking.rs:74:14
2: core::panicking::panic
at /rustc/90b35a6239c3d8bdabc530a6a0816f7ff89a0aaf/library/core/src/panicking.rs:148:5
3: core::option::unwrap_failed
at /rustc/90b35a6239c3d8bdabc530a6a0816f7ff89a0aaf/library/core/src/option.rs:2012:5
4: core::option::Option<T>::unwrap
at /Users/pedro/.rustup/toolchains/1.83.0-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/option.rs:972:21
5: limbo_core::storage:📟:Page::get_contents
at ./core/storage/pager.rs:61:9
6: limbo_core::storage::btree::BTreeCursor::balance_non_root
at ./core/storage/btree.rs:1723:40
7: limbo_core::storage::btree::BTreeCursor::balance
at ./core/storage/btree.rs:1570:35
8: limbo_core::storage::btree::BTreeCursor::insert_into_page
at ./core/storage/btree.rs:1512:35
9: limbo_core::storage::btree::BTreeCursor::insert
at ./core/storage/btree.rs:3024:31
10: limbo_core::vdbe::execute::op_insert
at ./core/vdbe/execute.rs:3654:23
11: limbo_core::vdbe::Program::step
at ./core/vdbe/mod.rs:379:23
12: limbo_core::Statement::step
at ./core/lib.rs:582:9
13: limbo::app::Limbo::print_query_result
at ./cli/app.rs:657:27
14: limbo::app::Limbo::run_query
at ./cli/app.rs:420:20
15: limbo::app::Limbo::handle_input_line
at ./cli/app.rs:527:13
16: limbo::main
at ./cli/main.rs:29:31
17: core::ops::function::FnOnce::call_once
at /Users/pedro/.rustup/toolchains/1.83.0-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
Testing: Tested large write to testfs
Test FAILED: Test failed
SQL: SELECT count(*) FROM test;
Actual:
None
make: *** [test-extensions] Error 1
```
Closes#1346
Sqlite reference implementation: https://github.com/sqlite/sqlite/blob/8
37dc09bce7de8971c7488b70cf5da93c60fbed0/src/vdbe.c#L2558
We did not support blobs before and our ordering of the match statements
were incorrect when one of the arguments was NULL
Closes#1337
Currently, to run the fuzz test you need to remove an assert statement.
EDIT: @diegoreis42 pointed it out to me that the method had just changed
actually. Made the change to use `row.len()` instead
# Problem
```
cargo +nightly fuzz run --target=aarch64-apple-darwin expression
Compiling limbo_core v0.0.19-pre.4 (/Users/pedro/Projects/limbo/core)
Compiling limbo-fuzz v0.0.0 (/Users/pedro/Projects/limbo/fuzz)
error[E0599]: `&limbo_core::vdbe::Row` is not an iterator
--> fuzz_targets/expression.rs:198:36
|
198 | assert_eq!(row.count(), 1, "expr: {:?}", expr);
| ^^^^^
| |
| `&limbo_core::vdbe::Row` is not an iterator
| private field, not a method
|
::: /Users/pedro/Projects/limbo/core/vdbe/mod.rs:221:1
|
221 | pub struct Row {
| -------------- doesn't satisfy `limbo_core::vdbe::Row: std::iter::Iterator`
|
= note: the following trait bounds were not satisfied:
`&limbo_core::vdbe::Row: std::iter::Iterator`
which is required by `&mut &limbo_core::vdbe::Row: std::iter::Iterator`
`limbo_core::vdbe::Row: std::iter::Iterator`
which is required by `&mut limbo_core::vdbe::Row: std::iter::Iterator`
For more information about this error, try `rustc --explain E0599`.
error: could not compile `limbo-fuzz` (bin "expression") due to 1 previous error
Error: failed to build fuzz script: ASAN_OPTIONS="detect_odr_violation=0" RUSTFLAGS="-Cpasses=sancov-module -Cllvm-args=-sanitizer-coverage-level=4 -Cllvm-args=-sanitizer-coverage-inline-8bit-counters -Cllvm-args=-sanitizer-coverage-pc-table -Cllvm-args=-sanitizer-coverage-trace-compares --cfg fuzzing -Clink-dead-code -Zsanitizer=address -Cdebug-assertions -C codegen-units=1" "cargo" "build" "--manifest-path" "/Users/pedro/Projects/limbo/fuzz/Cargo.toml" "--target" "aarch64-apple-darwin" "--release" "--config" "profile.release.debug=true" "--bin" "expression"
```
Closes#1334
Fixes#1298
- Fixes Limbo trying to use an index using a WHERE clause constraint
that refers to the same table on both sides, e.g. `WHERE t.x = t.x`
- Fixes not using indexes when the relevant expression is paren wrapped,
e.g.
- `SELECT * FROM t WHERE (indexcol) > 5`
- `SELECT * FROM t WHERE (indexcol > 5)`
- Changes existing table logical expr fuzz test to have primary keys
(which helped me find both issues above)
Closes#1300
Fixed is_empty assertion in #1203 , but simulator still has error.
```sh
[ERROR] error Internal error: select '(engrossing_berger < X'6566651A3C70278D4E200657551D8071A1' AND competitive_petit > 1236742147.9451914)' should return no values for table 'super_becky'
```
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#1308
Some enabler work for enabling autoindexing, which means creating a
transient in memory index when a persistent index is not found. This is
used to avoid nested full table scans.
@pereman2 I added some functionality to your `SmallVec` so would
appreciate a review of those parts
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#1340
First ~actual~ step to close#741, this time I'm following an approach
way closer to what SQLite does by just creating a normal `BTreeCursor`
and using an in-memory pager (thanks @krishvishal for the insight
🤝).
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#1139
