Commit Graph

2600 Commits

Author SHA1 Message Date
Pekka Enberg
36b487d281 core/mvcc: Make Clippy happy 2025-02-05 13:41:20 +02:00
Pekka Enberg
5870c92e9e core/mvcc: Fix MVCC benchmark SIGKILL
The `begin_tx` benchmark makes no sense because it just fills up memory with
transaction metadata, eventually killing the process...
2025-02-05 13:33:38 +02:00
Pekka Enberg
44ca85e121 core: Enable MVCC benchmark 2025-02-05 13:26:05 +02:00
Pekka Enberg
fad479ac59 core/mvcc: Move source code to module 2025-02-05 13:25:16 +02:00
Pekka Enberg
a585b81148 mvcc/core: Kill S3 persistent storage 2025-02-05 12:51:58 +02:00
Pekka Enberg
e923a2352e core/mvcc: Kill mvcc-rs crate
We'll just integrate everything in the core.
2025-02-05 12:50:46 +02:00
Pekka Enberg
9f0b33a8ef core/mvcc: Remove README.md 2025-02-05 12:50:46 +02:00
Pekka Enberg
5c9bb4bddd core/mvcc: Remove duplicate Cargo workspace config 2025-02-05 12:42:39 +02:00
Pekka Enberg
5e282c00bc Remove duplicate MIT license 2025-02-05 12:42:15 +02:00
Pekka Enberg
7d99894269 Move MVCC docs to top-level docs directory 2025-02-05 12:41:55 +02:00
Pekka Enberg
df20213a4b core/mvcc: Remove C bindings
We won't need them because we just use the Rust APIs in the core.
2025-02-05 12:40:28 +02:00
Pekka Enberg
fcb4c7e46a core/mvcc: Remove Git metadata files 2025-02-05 12:40:06 +02:00
Pekka Enberg
b9568b74af Merge "Hekaton MVCC implementation" from Pekka and others
This imports the full history of the following Git repository into
`core/mvcc` directory as-is:

https://github.com/penberg/tihku/tree/main
2025-02-05 12:38:35 +02:00
Pekka Enberg
9fdf54de2b Merge 'Small perf optimizations to statement preparation' from Jussi Saurio
```bash
Prepare `SELECT 1`/Limbo/SELECT 1
                        time:   [765.94 ns 768.26 ns 771.03 ns]
                        change: [-7.8340% -7.4887% -7.1406%] (p = 0.00 < 0.05)
                        Performance has improved.

Prepare `SELECT * FROM users LIMIT 1`/Limbo/SELECT * FROM users LIMIT 1
                        time:   [1.5673 µs 1.5699 µs 1.5731 µs]
                        change: [-10.810% -9.7122% -8.4951%] (p = 0.00 < 0.05)
                        Performance has improved.

Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou...
                        time:   [4.1331 µs 4.1421 µs 4.1513 µs]
                        change: [-9.3157% -9.0255% -8.7372%] (p = 0.00 < 0.05)
                        Performance has improved.
```
flamegraph for prepare `SELECT 1`:
<img width="1718" alt="Screenshot 2025-02-03 at 10 34 14"
src="https://github.com/user-
attachments/assets/ba67fe2f-78b2-4796-9a09-837d8e79fe62" />

Closes #872
2025-02-05 10:46:57 +02:00
Pekka Enberg
4aad309bab Merge 'clarify README and future plans' from Glauber Costa
Reviewed-by: Sonny (@sonhmai)

Closes #890
2025-02-05 10:46:34 +02:00
Pekka Enberg
64890f12e4 Merge 'core/vdbe: Lazy cursor borrowing' from Pekka Enberg
This saves a few more nanoseconds:
```
Execute `SELECT 1`/Limbo
                        time:   [44.964 ns 45.064 ns 45.160 ns]
                        change: [-14.371% -13.724% -13.214%] (p = 0.00 < 0.05)
                        Performance has improved.
```

Closes #892
2025-02-05 10:46:04 +02:00
Pekka Enberg
0b0681c9f8 core/vdbe: Lazy cursor borrowing
This saves a few more nanoseconds:

```
Execute `SELECT 1`/Limbo
                        time:   [44.964 ns 45.064 ns 45.160 ns]
                        change: [-14.371% -13.724% -13.214%] (p = 0.00 < 0.05)
                        Performance has improved.
```
2025-02-05 09:47:17 +02:00
Pekka Enberg
e3169e5dcd Merge 'Simplify Statement::step()' from Pekka Enberg
...brings back a few nanoseconds.

Closes #891
2025-02-05 09:26:04 +02:00
Pekka Enberg
23cd8b10c3 core: Unify StepResult structs
...also simplify Statement::step() to get some performance back.

Before:

```
Execute `SELECT 1`/Limbo
                        time:   [49.128 ns 50.425 ns 52.604 ns]
```

After:

```
Execute `SELECT 1`/Limbo
                        time:   [49.128 ns 50.425 ns 52.604 ns]
```
2025-02-05 09:09:32 +02:00
Pekka Enberg
7573fc62e6 core: Unify Row and Record structs
They're exactly the same thing.
2025-02-05 09:04:52 +02:00
Glauber Costa
ac6dc9f9e8 clarify README and future plans 2025-02-04 19:44:04 -05:00
Pekka Enberg
20ffcb9d48 Merge 'core: Switch to parking_lot for RwLock' from Pekka Enberg
We really need to make the WAL lock less expensive, but switching to
`parking_lot` is anyway something we should do.
Before:
```
Execute `SELECT 1`/Limbo
                        time:   [56.230 ns 56.463 ns 56.688 ns]
```
After:
```
Execute `SELECT 1`/Limbo
                        time:   [52.003 ns 52.132 ns 52.287 ns]
```

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #887
2025-02-04 20:55:54 +02:00
Pekka Enberg
1aca47a384 Replace cover art with Turso branding
I socially engineerd Jamie into generating a Turso branded cover image
by merging something he hated, which was AI-generated image of a dark
forest. Thanks Jamie!
2025-02-04 20:47:33 +02:00
Pekka Enberg
f004528e35 Update README.md 2025-02-04 20:07:21 +02:00
Pekka Enberg
68606d4d8c Update README.md cover art 2025-02-04 20:03:18 +02:00
Pekka Enberg
541270e38f Update README.md 2025-02-04 19:56:47 +02:00
Pekka Enberg
002ffb2e4d Update README.md 2025-02-04 19:35:07 +02:00
Pekka Enberg
4ad865af1d Merge 'Add Go example to main repo README' from Preston Thorpe
Now that we have a release that includes the Go bindings, we can have an
example where `go get github.com/tursodatabase/limbo` is used 👍

Closes #888
2025-02-04 19:31:47 +02:00
PThorpe92
1cbd33499e Add Go example to README 2025-02-04 11:53:11 -05:00
Pekka Enberg
e4d7474372 core: Switch to parking_lot for RwLock
We really need to make the WAL lock less expensive, but switching to
`parking_lot` is anyway something we should do.

Before:

```
Execute `SELECT 1`/Limbo
                        time:   [56.230 ns 56.463 ns 56.688 ns]
```

After:

```
Execute `SELECT 1`/Limbo
                        time:   [52.003 ns 52.132 ns 52.287 ns]
```
2025-02-04 18:38:33 +02:00
Pekka Enberg
750164fb85 Merge 'Sqlean Time extension' from Pedro Muniz
This PR implements a sqlean time compatible extension. I would
appreciate some help to review my code and see if there are ways to
enhance it. Also, if there is some edge case, I have missed please tell
me.
https://github.com/nalgeon/sqlean/blob/main/docs/time.md

Closes #854
2025-02-04 18:27:44 +02:00
Pekka Enberg
733f7de688 Merge branch 'main' into feature/time-ext 2025-02-04 18:27:14 +02:00
Pekka Enberg
f69804969c Merge 'Adding checkpoint result' from Sonny
### What?
adding checkpoint result returning number of pages in wal and num pages
checkpointed.
Part of #696
### Context
SQLite returns in checkpoint result of calling `pragma wal_checkpoint;`
`0|3|3` while limbo returns `0|0|0`.
https://sqlite.org/pragma.html#pragma_wal_checkpoint
- 1st col: 1 (checkpoint SQLITE_BUSY) or 0 (not busy).
- 2nd col: # modified pages written to wal file
- 3rd col: # pages moved to db after checkpoint
This PR aims to add 2nd and 3rd column to the checkpoint result.
SQLite
```
sqlite3 test.db
sqlite> pragma journal_mode=wal;
wal
sqlite> pragma journal_mode;
wal
sqlite> create table t1 (id text);
sqlite> insert into t1(id) values (1),(2);
sqlite> select * from t1;
1
2
sqlite> pragma wal_checkpoint;
0|3|3
```
Limbo
```
./target/debug/limbo test.db
Limbo v0.0.13
Enter ".help" for usage hints.
limbo> pragma journal_mode;
wal
limbo> create table t1(id text);
limbo> insert into t1(id) values (1),(2);
limbo> select * from t1;
1
2
# current the 2nd and 3rd columns are hard coded in limbo to 0
limbo> pragma wal_checkpoint;
0|0|0
```

Closes #827
2025-02-04 18:26:24 +02:00
Pekka Enberg
a045866a6c Merge 'core: Fix benchmark panic if syscall is interrupted' from Pekka Enberg
Fixes the following panics:
Benchmarking Execute `SELECT * FROM users LIMIT ?`/Limbo/100: Profiling
for 5.0000 sthread 'main' panicked at core/benches/benchmark.rs:69:43:
called `Result::unwrap()` on an `Err` value: IOError(Os { code: 4, kind:
Interrupted, message: "Interrupted system call" })
note: run with `RUST_BACKTRACE=1` environment variable to display a
backtrace

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #882
2025-02-04 18:25:34 +02:00
Pekka Enberg
b20ad42892 Merge 'github: Fix Python release workflow' from Pekka Enberg
We need to switch to `download-artifact@v4` because v3 is dead...

Closes #883
2025-02-04 18:25:26 +02:00
Pekka Enberg
bf1ef13c91 Merge 'Add Printf Support' from Zaid Humayun
Add basic printf function support in limbo
![Screenshot 2025-02-04 at 8 08 23 PM](https://github.com/user-
attachments/assets/b12931eb-8e79-4c8a-af77-c25c34cc5834)

Closes #886
2025-02-04 17:53:27 +02:00
Zaid Humayun
15b46e73eb test fix 2025-02-04 21:02:51 +05:30
Zaid Humayun
39d57ba541 test fix again 2025-02-04 20:58:03 +05:30
Zaid Humayun
4cb6f8cc6a fixing: failing test was expecting 2.50000
this test is behaving differently in different environments
2025-02-04 20:53:10 +05:30
Zaid Humayun
6a863b3da9 printf: this commit adds support for https://github.com/tursodatabase/limbo/issues/885 tracking printf functionality
this commit introduces basic support for printf functionality and doesn't include advanced modifiers like width etc.
2025-02-04 20:05:14 +05:30
Pekka Enberg
1817b75d9b github: Fix Python release workflow
We need to switch to `download-artifact@v4` because v3 is dead...
2025-02-04 15:55:19 +02:00
Pekka Enberg
73a345aa4d Limbo 0.0.14 2025-02-04 15:13:18 +02:00
Pekka Enberg
31552f0862 core/json: Kill some unwrap() calls 2025-02-04 14:55:19 +02:00
Pekka Enberg
2d9a54ccba Merge 'implement json_pretty' from Pedro Muniz
This PR implements json_pretty. At the moment, support for jsonb is
being added, so this function suffers from the same limitations as in
json(x). Also, I have not found a way to implement the same conversion
of Blob -> String that SQLite does. From my own experimentation, I
believe SQLite converts blobs to a lossy ascii representation, but I
would appreciate some help on this.

Closes #860
2025-02-04 14:51:47 +02:00
Pekka Enberg
0050f4aeea Merge 'Support column aliases in GROUP BY, ORDER BY and HAVING' from Jussi Saurio
Closes #744
```sql
# Wanda = 9, Whitney = 11, William = 111
do_execsql_test column_alias_in_group_by_order_by_having {
  select first_name as fn, count(1) as fn_count from users where fn in ('Wanda', 'Whitney', 'William') group by fn having fn_count > 10 order by fn_count;
} {Whitney|11
William|111}
```

Closes #864
2025-02-04 14:51:05 +02:00
Pekka Enberg
b4a3cadf58 Merge 'Improve in-memory IO performance, remove runtime borrow checking' from Preston Thorpe
Despite likely replacing this in-memory IO setup in the near future with
a `mmap` implementation (#859) , in the spirit of everyone getting
bitten by the perf bug lately I thought I would speed up our in-memory
IO a bit.

Closes #861
2025-02-04 14:50:38 +02:00
Pekka Enberg
e2cf2d2c37 Merge 'doc: refine function contributing doc' from Sonny
improved function contributing doc a bit.

Closes #873
2025-02-04 14:50:11 +02:00
Pekka Enberg
1a1366e46b Merge 'core: Kill Statement::query() method' from Pekka Enberg
It's a pointless wrapper on top of `step()` that introduce additional
memory allocation and deallocation.

Closes #880
2025-02-04 14:01:57 +02:00
Pekka Enberg
097e56c19f core: Kill Statement::query() method
It's a pointless wrapper on top of `step()` that introduce additional
memory allocation and deallocation.
2025-02-04 13:46:11 +02:00
Pekka Enberg
7dbfdba59f core: Fix benchmark panic if syscall is interrupted
Fixes the following panics:

Benchmarking Execute `SELECT * FROM users LIMIT ?`/Limbo/100: Profiling for 5.0000 sthread 'main' panicked at core/benches/benchmark.rs:69:43:
called `Result::unwrap()` on an `Err` value: IOError(Os { code: 4, kind: Interrupted, message: "Interrupted system call" })
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
2025-02-04 13:24:55 +02:00