This PR changes the argument parsing and matching to use Clap instead of
our own handrolled one. This makes it much easier to add new commands
and modify existing ones by using the full power of clap's derive
macros. It also produces nice error and help messages for us. However,
there is a bug in Clap that is not correctly modifying the `display
name` in the help section. So the command would appear as `show` instead
of `.show` in the help messages. This is very minimal, but if this is a
blocker for this PR we can just overwrite the help message in its
entirety. Also using Clap would enable us to use its autocomplete crate
to generate autocompletions for these special repl commands which would
be a huge win when compared to the `sqlite3` cli.
This is the current help message:
```sh
Limbo SQL Shell Help
==============
Welcome to the Limbo SQL Shell! You can execute any standard SQL command here.
In addition to standard SQL commands, the following special commands are available:
Usage: <COMMAND>
Commands:
exit Exit this program with return-code CODE
quit Quit the shell
open Open a database file
schema Print this message or the help of the given subcommand(s) Display schema for a table
output Set output file (or stdout if empty)
mode Set output display mode
opcodes Show vdbe opcodes
cd Change the current working directory
show Display information about settings
nullvalue Set the value of NULL to be printed in 'list' mode
echo Toggle 'echo' mode to repeat commands before execution
tables Display tables
import Import data from FILE into TABLE
load Loads an extension library
dump Dump the current database as a list of SQL statements
listvfs List vfs modules available
help Print this message or the help of the given subcommand(s)
Usage Examples:
---------------
1. To quit the Limbo SQL Shell:
.quit
2. To open a database file at path './employees.db':
.open employees.db
3. To view the schema of a table named 'employees':
.schema employees
4. To list all tables:
.tables
5. To list all available SQL opcodes:
.opcodes
6. To change the current output mode to 'pretty':
.mode pretty
7. Send output to STDOUT if no file is specified:
.output
8. To change the current working directory to '/tmp':
.cd /tmp
9. Show the current values of settings:
.show
10. To import csv file 'sample.csv' into 'csv_table' table:
.import --csv sample.csv csv_table
11. To display the database contents as SQL:
.dump
12. To load an extension library:
.load /target/debug/liblimbo_regexp
13. To list all available VFS:
.listvfs
Note:
- All SQL commands must end with a semicolon (;).
- Special commands start with a dot (.) and are not required to end with a semicolon.
```
If we need more information on a specific command, we can leverage CLAP
and do for instance:
```sh
.open -h
```
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#1110
We need to ensure an operation doesn't transform the cells inside a page
to an invalid state. In debug mode we can enable a
`debug_validate_cells` with `#[cfg(debug_assertions)]` so that it is
skipped on release mode.
Modify pager logs
Newly added divider cells to parent of an interior page must point to
the page in question. Moreover rightmost pointer of the page will point
to previous divider cell pointer.
Closes#1221
Newly added divider cells to parent of an interior page must point to
the page in question. Moreover rightmost pointer of the page will point
to previous divider cell pointer.
Also I added more tests for exec_add, exec_subtract, exec_multiply,
exec_divide, exec_remainder.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#1214
We never hit bugs due to these because of 1. not having multi column
indexes in our TCL test databases, 2. otherwise not really having Rust
tests involving indexes, and 3. `IdxLt` and `IdxLe` not actually being
used anywhere yet
Also as @PThorpe92 pointed out there are some nuances to the comparison
logic we may need to eventually implement regarding comparisons with
uneven number of keys:
https://github.com/sqlite/sqlite/blob/master/src/vdbeaux.c#L4719
Reviewed-by: Preston Thorpe (@PThorpe92)
Closes#1215
1. Removed dependecy on serde, now all json functions use jsonb
implementation.
2. Add support for jsonb_patch.
3. Aggregation json functions like json_group_array, json_group_object.
Closes#1207
This pr is huge again but I will try to introduce each improvement one
by one.
## Overview
### Remove Rc for Text and Blob.
In general copying is bad, that's why we hid it with `Rc`s. With the
introduction of `ImmutableRecord` we make it less relevant because now
we will copy only once anyways, no other place should copy it so we can
avoid using `Rc`. If we we were to copy it it most likely means where
are doing something wrong.
### Reuse `Text` and `Blob` OwnedValues.
Most of the queries spend time overwriting the same register over and
over. What about we don't allocate new `OwnedValue` and we just simply
reuse the `OwnedValue` and extend the internal buffer. That's what I did
and it worked quite nicely.
### Make `Register::Record` be `ImmutableRecord`
`ImmutableRecord` basically means "serialized record", that's why all
the data is contained in a single payload buffer. There is a list of
values to reference that payload to reduce time complexity of search --
there is an argument to make a record without this vec to reduce memory
footprint. This improvement I don't think it had a direct impact on
performance but it is a simpler way to lay the memory without any
complicated reference counted pointers, and instead we use a contiguous
piece of memory.
### Make `ImmutableRecord` reusable in `BTreeCursor`.
`BTreeCursor` allocated and deallocated records when it needed a new
one. This is obviously a big waste because we could be reusing the
internal buffer to avoid allocations. `ImmutableRecord` proved to be
useful here because now, we will only store a single `ImmutableRecord`
in the cursor that we will never deallocate -- we will just reallocate
when needed and replace the current one with the next one on demand.
## Return `Row` as a reference of Registers.
A `ResultRow` bytecode takes care of gathering all the columns of a row
and returning them to the user. Previously we could create a new
`Record` struct with all the cloned values which proved to be wasteful.
SQLite is smart about this so we must be as well. Basically a row now is
a wrapper for `struct Row { *const Register, count: usize }`, and we
basically include some QOL methods to avoid using pointers directly.
I know pointers are unsafe. That's why this row will be invalidate on
the next step of the VM and this row should be not used outside there.
### Inlining go brrr
`read_varint` and `read_value` are called in a tight loop making it easy
to see overhead of the call stack. That's why I sprinkled some
`#[inline(always)]` and saw something like a 15% speed boost.
## read_record with custom `SmallVec<T>`
We tend to overuse vectors for everything, this is quite bad because it
requires heap memory allocations. We can avoid this with a simple
`SmallVec` that simply fallsback to a vec with more complex scenarios.
## Benchmarks!
```
### before
fun/limbo » cargo bench -- limbo_execute 2>&1 | grep -B 1 "time: " | tee out.log
Execute `SELECT 1`/limbo_execute_select_1
time: [43.958 ns 44.056 ns 44.154 ns]
--
Execute `SELECT * FROM users LIMIT ?`/limbo_execute_select_rows/1
time: [407.82 ns 408.57 ns 409.41 ns]
--
Execute `SELECT * FROM users LIMIT ?`/limbo_execute_select_rows/10
time: [2.7335 µs 2.7386 µs 2.7443 µs]
--
Execute `SELECT * FROM users LIMIT ?`/limbo_execute_select_rows/50
time: [13.451 µs 13.485 µs 13.520 µs]
--
Execute `SELECT * FROM users LIMIT ?`/limbo_execute_select_rows/100
time: [26.967 µs 27.077 µs 27.201 µs]after:
```
### after
```
fun/limbo (more-register) » cargo bench -- limbo_execute 2>&1 | grep -B 1 "time: " | tee out.log 130 ↵
Execute `SELECT 1`/limbo_execute_select_1
time: [33.386 ns 33.440 ns 33.510 ns]
--
Execute `SELECT * FROM users LIMIT ?`/limbo_execute_select_rows/1
time: [326.79 ns 327.37 ns 328.03 ns]
--
Execute `SELECT * FROM users LIMIT ?`/limbo_execute_select_rows/10
time: [1.5817 µs 1.5849 µs 1.5889 µs]
--
Execute `SELECT * FROM users LIMIT ?`/limbo_execute_select_rows/50
time: [7.3295 µs 7.3531 µs 7.3829 µs]
--
Execute `SELECT * FROM users LIMIT ?`/limbo_execute_select_rows/100
time: [14.538 µs 14.570 µs 14.606 µs]
```
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#1197
