3e28541b53
In limbo when we do any compare operations like `Eq, gt, lt, gte, lte` with nulls , we were actually giving the result as true where as sqlite3 gives null. This is because if we had a null, we were incorrectly going to conditional branch and not increment program by 1. Also the sqlite generates `ZeroOrNull` op in these cases (https://github.com/sqlite/sqlite/blob/version-3.45.3/src/expr.c#L4644) but we were generating a Integer instruction. The below outputs can give a clearer picture. This PR aims to fix this. sqlite3 output ``` SQLite version 3.48.0 Enter ".help" for usage hints. Connected to a transient in-memory database. Use ".open FILENAME" to reopen on a persistent database. sqlite> select 8 = null; sqlite> select 8 > null; sqlite> explain select 8 > null; addr opcode p1 p2 p3 p4 p5 comment ---- ------------- ---- ---- ---- ------------- -- ------------- 0 Init 0 6 0 0 1 Integer 1 1 0 0 2 Gt 3 4 2 64 3 ZeroOrNull 2 1 3 0 4 ResultRow 1 1 0 0 5 Halt 0 0 0 0 6 Integer 8 2 0 0 7 Null 0 3 0 0 8 Goto 0 1 0 0 sqlite> explain select 8 = null; addr opcode p1 p2 p3 p4 p5 comment ---- ------------- ---- ---- ---- ------------- -- ------------- 0 Init 0 6 0 0 1 Integer 1 1 0 0 2 Eq 3 4 2 64 3 ZeroOrNull 2 1 3 0 4 ResultRow 1 1 0 0 5 Halt 0 0 0 0 6 Integer 8 2 0 0 7 Null 0 3 0 0 8 Goto 0 1 0 0 ``` Limbo Output ``` Limbo v0.0.12 Enter ".help" for usage hints. Connected to a transient in-memory database. Use ".open FILENAME" to reopen on a persistent database limbo> select 8 = null; 1 limbo> select 8 > null; 1 limbo> explain select 8 > null; addr opcode p1 p2 p3 p4 p5 comment ---- ----------------- ---- ---- ---- ------------- -- ------- 0 Init 0 8 0 0 Start at 8 1 Integer 8 2 0 0 r[2]=8 2 Null 0 3 0 0 r[3]=NULL 3 Integer 1 1 0 0 r[1]=1 4 Gt 2 3 6 0 if r[2]>r[3] goto 6 5 Integer 0 1 0 0 r[1]=0 6 ResultRow 1 1 0 0 output=r[1] 7 Halt 0 0 0 0 8 Transaction 0 0 0 0 9 Goto 0 1 0 0 limbo> explain select 8 = null; addr opcode p1 p2 p3 p4 p5 comment ---- ----------------- ---- ---- ---- ------------- -- ------- 0 Init 0 8 0 0 Start at 8 1 Integer 8 2 0 0 r[2]=8 2 Null 0 3 0 0 r[3]=NULL 3 Integer 1 1 0 0 r[1]=1 4 Eq 2 3 6 0 if r[2]==r[3] goto 6 5 Integer 0 1 0 0 r[1]=0 6 ResultRow 1 1 0 0 output=r[1] 7 Halt 0 0 0 0 8 Transaction 0 0 0 0 9 Goto 0 1 0 0 limbo> ``` Limbo Output with this PR ``` Limbo v0.0.12 Enter ".help" for usage hints. Connected to a transient in-memory database. Use ".open FILENAME" to reopen on a persistent database limbo> select 8 = null; limbo> select 8 > null; limbo> explain select 8 > null; addr opcode p1 p2 p3 p4 p5 comment ---- ----------------- ---- ---- ---- ------------- -- ------- 0 Init 0 8 0 0 Start at 8 1 Integer 8 2 0 0 r[2]=8 2 Null 0 3 0 0 r[3]=NULL 3 Integer 1 1 0 0 r[1]=1 4 Gt 2 3 6 0 if r[2]>r[3] goto 6 5 ZeroOrNull 2 1 3 0 ((r[2]=NULL)|(r[3]=NULL)) ? r[1]=NULL : r[1]=0 6 ResultRow 1 1 0 0 output=r[1] 7 Halt 0 0 0 0 8 Transaction 0 0 0 0 9 Goto 0 1 0 0 limbo> explain select 8 = null; addr opcode p1 p2 p3 p4 p5 comment ---- ----------------- ---- ---- ---- ------------- -- ------- 0 Init 0 8 0 0 Start at 8 1 Integer 8 2 0 0 r[2]=8 2 Null 0 3 0 0 r[3]=NULL 3 Integer 1 1 0 0 r[1]=1 4 Eq 2 3 6 0 if r[2]==r[3] goto 6 5 ZeroOrNull 2 1 3 0 ((r[2]=NULL)|(r[3]=NULL)) ? r[1]=NULL : r[1]=0 6 ResultRow 1 1 0 0 output=r[1] 7 Halt 0 0 0 0 8 Transaction 0 0 0 0 9 Goto 0 1 0 0 ``` Closes #733
Limbo
Limbo is a work-in-progress, in-process OLTP database management system, compatible with SQLite.
Features
Limbo is an in-process OLTP database engine library that has:
- Asynchronous I/O support on Linux with
io_uring - SQLite compatibility [doc] for SQL dialect, file formats, and the C API
- Language bindings for JavaScript/WebAssembly, Rust, Python, and Java
- OS support for Linux, macOS, and Windows
Getting Started
CLI
Install limbo with:
curl --proto '=https' --tlsv1.2 -LsSf \
https://github.com/tursodatabase/limbo/releases/latest/download/limbo-installer.sh | sh
Then use the SQL shell to create and query a database:
$ limbo database.db
Limbo v0.0.6
Enter ".help" for usage hints.
limbo> CREATE TABLE users (id INT PRIMARY KEY, username TEXT);
limbo> INSERT INTO users VALUES (1, 'alice');
limbo> INSERT INTO users VALUES (2, 'bob');
limbo> SELECT * FROM users;
1|alice
2|bob
JavaScript (wip)
Installation:
npm i limbo-wasm
Example usage:
import { Database } from 'limbo-wasm';
const db = new Database('sqlite.db');
const stmt = db.prepare('SELECT * FROM users');
const users = stmt.all();
console.log(users);
Python (wip)
pip install pylimbo
Example usage:
import limbo
con = limbo.connect("sqlite.db")
cur = con.cursor()
res = cur.execute("SELECT * FROM users")
print(res.fetchone())
Developing
Build and run limbo cli:
cargo run --package limbo --bin limbo database.db
Run tests:
cargo test
Test coverage report:
cargo tarpaulin -o html
Note
Generation of coverage report requires tarpaulin binary to be installed. You can install it with
cargo install cargo-tarpaulin
Tip
If coverage fails with "Test failed during run" error and all of the tests passed it might be the result of tarpaulin bug. You can temporarily set dynamic libraries linking manually as a workaround, e.g. for linux
LD_LIBRARY_PATH="$(rustc --print=target-libdir)" cargo tarpaulin -o html.
Run benchmarks:
cargo bench
Run benchmarks and generate flamegraphs:
echo -1 | sudo tee /proc/sys/kernel/perf_event_paranoid
cargo bench --bench benchmark -- --profile-time=5
FAQ
How is Limbo different from libSQL?
Limbo is a research project to build a SQLite compatible in-process database in Rust with native async support. The libSQL project, on the other hand, is an open source, open contribution fork of SQLite, with focus on production features such as replication, backups, encryption, and so on. There is no hard dependency between the two projects. Of course, if Limbo becomes widely successful, we might consider merging with libSQL, but that is something that will be decided in the future.
Publications
- Pekka Enberg, Sasu Tarkoma, Jon Crowcroft Ashwin Rao (2024). Serverless Runtime / Database Co-Design With Asynchronous I/O. In EdgeSys ‘24. [PDF]
- Pekka Enberg, Sasu Tarkoma, and Ashwin Rao (2023). Towards Database and Serverless Runtime Co-Design. In CoNEXT-SW ’23. [PDF] [Slides]
Contributing
We'd love to have you contribute to Limbo! Check out the contribution guide to get started.
License
This project is licensed under the MIT license.
Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in Limbo by you, shall be licensed as MIT, without any additional terms or conditions.