We really need to make the WAL lock less expensive, but switching to
`parking_lot` is anyway something we should do.
Before:
```
Execute `SELECT 1`/Limbo
time: [56.230 ns 56.463 ns 56.688 ns]
```
After:
```
Execute `SELECT 1`/Limbo
time: [52.003 ns 52.132 ns 52.287 ns]
```
This PR implements a sqlean time compatible extension. I would
appreciate some help to review my code and see if there are ways to
enhance it. Also, if there is some edge case, I have missed please tell
me.
https://github.com/nalgeon/sqlean/blob/main/docs/time.mdCloses#854
### What?
adding checkpoint result returning number of pages in wal and num pages
checkpointed.
Part of #696
### Context
SQLite returns in checkpoint result of calling `pragma wal_checkpoint;`
`0|3|3` while limbo returns `0|0|0`.
https://sqlite.org/pragma.html#pragma_wal_checkpoint
- 1st col: 1 (checkpoint SQLITE_BUSY) or 0 (not busy).
- 2nd col: # modified pages written to wal file
- 3rd col: # pages moved to db after checkpoint
This PR aims to add 2nd and 3rd column to the checkpoint result.
SQLite
```
sqlite3 test.db
sqlite> pragma journal_mode=wal;
wal
sqlite> pragma journal_mode;
wal
sqlite> create table t1 (id text);
sqlite> insert into t1(id) values (1),(2);
sqlite> select * from t1;
1
2
sqlite> pragma wal_checkpoint;
0|3|3
```
Limbo
```
./target/debug/limbo test.db
Limbo v0.0.13
Enter ".help" for usage hints.
limbo> pragma journal_mode;
wal
limbo> create table t1(id text);
limbo> insert into t1(id) values (1),(2);
limbo> select * from t1;
1
2
# current the 2nd and 3rd columns are hard coded in limbo to 0
limbo> pragma wal_checkpoint;
0|0|0
```
Closes#827
Fixes the following panics:
Benchmarking Execute `SELECT * FROM users LIMIT ?`/Limbo/100: Profiling
for 5.0000 sthread 'main' panicked at core/benches/benchmark.rs:69:43:
called `Result::unwrap()` on an `Err` value: IOError(Os { code: 4, kind:
Interrupted, message: "Interrupted system call" })
note: run with `RUST_BACKTRACE=1` environment variable to display a
backtrace
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#882
Add basic printf function support in limbo
Closes#886
This PR implements json_pretty. At the moment, support for jsonb is
being added, so this function suffers from the same limitations as in
json(x). Also, I have not found a way to implement the same conversion
of Blob -> String that SQLite does. From my own experimentation, I
believe SQLite converts blobs to a lossy ascii representation, but I
would appreciate some help on this.
Closes#860
Closes#744
```sql
# Wanda = 9, Whitney = 11, William = 111
do_execsql_test column_alias_in_group_by_order_by_having {
select first_name as fn, count(1) as fn_count from users where fn in ('Wanda', 'Whitney', 'William') group by fn having fn_count > 10 order by fn_count;
} {Whitney|11
William|111}
```
Closes#864
Despite likely replacing this in-memory IO setup in the near future with
a `mmap` implementation (#859) , in the spirit of everyone getting
bitten by the perf bug lately I thought I would speed up our in-memory
IO a bit.
Closes#861
Fixes the following panics:
Benchmarking Execute `SELECT * FROM users LIMIT ?`/Limbo/100: Profiling for 5.0000 sthread 'main' panicked at core/benches/benchmark.rs:69:43:
called `Result::unwrap()` on an `Err` value: IOError(Os { code: 4, kind: Interrupted, message: "Interrupted system call" })
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
We spend a lot of time especially in `GROUP BY` queries providing
helpful comments for `EXPLAIN`, even when the query is not an `EXPLAIN`.
So let's not do that
Closes#784
```sql
Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou...
time: [4.2724 µs 4.2783 µs 4.2848 µs]
change: [-6.1063% -5.7376% -5.3626%] (p = 0.00 < 0.05)
Performance has improved.
```
doesn't affect the other trivial prepare benchmarks
Closes#875
If you have a `&str` you would need to allocate and copy the string just
to pass a reference to it again. Same goes if you have a slice of bytes.
In all (most?) situations, that is not what you want and sometimes
impossible to satisfy. Example:
```rs
impl From<&str> for Value<'_> {
fn from(value: &str) -> Self {
Self::Text(&value.to_owned())
}
}
```
Here, there is no way to pass a reference to a `String` without making
the lifetime `'static`, since the string has to be dropped by the end of
the function or leaked. I would consider this a anti-pattern. There is
no reason to keep a shared reference to a owned value. (And can't think
of any situation where you would need such thing)
Now, this is possible:
```rs
impl<'a> From<&'a str> for Value<'a> {
fn from(value: &'a str) -> Self {
Self::Text(value)
}
}
```
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Closes#838
Fix few logical codegen issues and add fuzz tests for logical
expressions
- Right now Limbo fails to recognize `false` constant in case when any
unary operator is used on the AST path. This PR add unary operator
option in the rewrite code and handle such cases properly.
```sql
limbo> SELECT NOT FALSE;
× Parse error: no such column: FALSE - should this be a string literal in single-quotes?
```
- `ifnull` implementation produced incorrect codegen due to "careless"
management of registers
```
limbo> SELECT ifnull(0, NOT 0)
[NULL here]
```
- `like` implementation produced incorrect codegen due to "careless"
management of registers
```
limbo> SELECT like('a%', 'a') = 1;
thread 'main' panicked at core/vdbe/mod.rs:1902:41:
internal error: entered unreachable code: Like on non-text registers
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
Depends on https://github.com/tursodatabase/limbo/pull/867 (need
`GrammarGenerator` from this branch)
Closes#869
This PR fixes a bug when index search used incorrect operator if index
column were the "rhs" in the expression (not "lhs" as usual, e.g.
`SELECT * FROM t WHERE 1 < rowid_alias`)
Reviewed-by: Jussi Saurio (@jussisaurio)
Closes#870
This PR fixes implementation of binary shift right/left instructions.
Before there were a minor incompatibility between limbo and sqlite
implementation in case when right shift second argument were more than
64 and first argument were negative. As sqlite implementation of right
binary shift is sign-extended - so `-1` should be returned in such case
when limbo returned zero.
This PR fixes this bug and also introduce a fuzz tests for arithemtic
expressions. This fuzz test were written with a help of
`GrammarGenerator` which allows to easily define probabilistic context-
free grammar and then later sample random strings from it.
Closes#867
# The purpose of this PR
- Current implementation forces users to use logback as their logging
framework
# Changes
- Only add abstraction layer for loggin(which is slf4j in this case)
- In tests, use logback to log out messages(this doesn't affect the
users)
# References
https://github.com/tursodatabase/limbo/issues/615Closes#863
# Purpose of this PR
- When loading native libraries, search jar path as well
- When packaging into a jar file, add native libraries under `/lib`
## How sqlite-jdbc packages their jar file
> Our SQLiteJDBC library requires no configuration since native
libraries for major OSs, including Windows, macOS, Linux etc., are
assembled into a single JAR (Java Archive) file.
=> sqlite-jdbc also packages all major OSs native libraries into a
single JAR
# Changes
- Add build commands in Makefile to build and publish locally
- Load native libraries from `/lib` under jar path
- Add java example under `/example`
# TODO
- Publish to maven central. I might need some help from maintainers.
- We can do better by not adding all the native libraries for every OSs
into the jar(as things can get big, though in compared to JVM, it's
relatively small). We can build for independent OSs and upload the
native libraries somewhere and let users download it and place the
native libraries under system path(which their java apps can discover).
- Or maybe introduce AOT(not sure how to)
# Reference
- [Issue](https://github.com/tursodatabase/limbo/issues/615)
- Now we can do something like this
Closes#862
Fix panic in case when table has row with rowid equals to `-1`
(`=u64::max`)
```sql
limbo> CREATE TABLE t(x INTEGER PRIMARY KEY)
limbo> INSERT INTO t VALUES (-1)
limbo> INSERT INTO t VALUES (NULL);
thread 'main' panicked at core/vdbe/mod.rs:2499:21:
attempt to add with overflow
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Closes#868
We transform all JOIN conditions into WHERE clause terms in the query
planner. The JoinAwareConditionExpr name tries to make that point, but I
think it makes things more confusing. Let's call it WhereTerm (suggested
by Jussi).
