Commit Graph

2606 Commits

Author SHA1 Message Date
Pekka Enberg
f772fc83e1 core/mvcc: Disable test_overlapping_concurrent_inserts_read_your_writes test
...it fails sporadically
2025-02-05 14:18:56 +02:00
Pekka Enberg
56d401fb67 Merge 'Implement json_set' from Marcus Nilsson
This PR adds support for `json_set`.
There are three helper functions added:
1. `json_path_from_owned_value`, this function turns an `OwnedValue`
into a `JsonPath`.
2. `find_or_create_target`, this function is similar to `find_target`
with the added bonus of creating the target if it doesn't exist. There
is a caveat with this function and that is that it will create
objects/arrays as it goes, meaning if you send `{}` into it and try
getting the path `$.some.nested.array[123].field`, it will return
`{"some":{"nested":array:[]}}` since creation of `some`, `nested` and
`array` will succeed, but accessing element `123` will fail.
3. `create_and_mutate_json_by_path`, this function is very similar to
`mutate_json_by_path` but calls `find_or_create_target` instead of
`find_target`

Related to #127

Closes #878
2025-02-05 14:15:02 +02:00
Pekka Enberg
f69b166d80 Merge 'Import MVCC implementation to the source tree' from Pekka Enberg
This pulls the MVCC implementation based on Hekaton me, @psarna, and
@avinassh did two years ago. No need to get excited, it's not integrated
to the Limbo core, just adds bunch of code likely full of bugs. Next
step is to see how to integrate this under a feature flag to query
executor and wire up transaction log to emit WAL.

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #896
2025-02-05 14:00:24 +02:00
Pekka Enberg
acb98f56d5 core/mvcc: Thanks Clippy... 2025-02-05 13:44:55 +02:00
Pekka Enberg
36b487d281 core/mvcc: Make Clippy happy 2025-02-05 13:41:20 +02:00
Pekka Enberg
5870c92e9e core/mvcc: Fix MVCC benchmark SIGKILL
The `begin_tx` benchmark makes no sense because it just fills up memory with
transaction metadata, eventually killing the process...
2025-02-05 13:33:38 +02:00
Pekka Enberg
44ca85e121 core: Enable MVCC benchmark 2025-02-05 13:26:05 +02:00
Pekka Enberg
fad479ac59 core/mvcc: Move source code to module 2025-02-05 13:25:16 +02:00
Pekka Enberg
a585b81148 mvcc/core: Kill S3 persistent storage 2025-02-05 12:51:58 +02:00
Pekka Enberg
e923a2352e core/mvcc: Kill mvcc-rs crate
We'll just integrate everything in the core.
2025-02-05 12:50:46 +02:00
Pekka Enberg
9f0b33a8ef core/mvcc: Remove README.md 2025-02-05 12:50:46 +02:00
Pekka Enberg
5c9bb4bddd core/mvcc: Remove duplicate Cargo workspace config 2025-02-05 12:42:39 +02:00
Pekka Enberg
5e282c00bc Remove duplicate MIT license 2025-02-05 12:42:15 +02:00
Pekka Enberg
7d99894269 Move MVCC docs to top-level docs directory 2025-02-05 12:41:55 +02:00
Pekka Enberg
df20213a4b core/mvcc: Remove C bindings
We won't need them because we just use the Rust APIs in the core.
2025-02-05 12:40:28 +02:00
Pekka Enberg
fcb4c7e46a core/mvcc: Remove Git metadata files 2025-02-05 12:40:06 +02:00
Pekka Enberg
b9568b74af Merge "Hekaton MVCC implementation" from Pekka and others
This imports the full history of the following Git repository into
`core/mvcc` directory as-is:

https://github.com/penberg/tihku/tree/main
2025-02-05 12:38:35 +02:00
Pekka Enberg
9fdf54de2b Merge 'Small perf optimizations to statement preparation' from Jussi Saurio
```bash
Prepare `SELECT 1`/Limbo/SELECT 1
                        time:   [765.94 ns 768.26 ns 771.03 ns]
                        change: [-7.8340% -7.4887% -7.1406%] (p = 0.00 < 0.05)
                        Performance has improved.

Prepare `SELECT * FROM users LIMIT 1`/Limbo/SELECT * FROM users LIMIT 1
                        time:   [1.5673 µs 1.5699 µs 1.5731 µs]
                        change: [-10.810% -9.7122% -8.4951%] (p = 0.00 < 0.05)
                        Performance has improved.

Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou...
                        time:   [4.1331 µs 4.1421 µs 4.1513 µs]
                        change: [-9.3157% -9.0255% -8.7372%] (p = 0.00 < 0.05)
                        Performance has improved.
```
flamegraph for prepare `SELECT 1`:
<img width="1718" alt="Screenshot 2025-02-03 at 10 34 14"
src="https://github.com/user-
attachments/assets/ba67fe2f-78b2-4796-9a09-837d8e79fe62" />

Closes #872
2025-02-05 10:46:57 +02:00
Pekka Enberg
4aad309bab Merge 'clarify README and future plans' from Glauber Costa
Reviewed-by: Sonny (@sonhmai)

Closes #890
2025-02-05 10:46:34 +02:00
Pekka Enberg
64890f12e4 Merge 'core/vdbe: Lazy cursor borrowing' from Pekka Enberg
This saves a few more nanoseconds:
```
Execute `SELECT 1`/Limbo
                        time:   [44.964 ns 45.064 ns 45.160 ns]
                        change: [-14.371% -13.724% -13.214%] (p = 0.00 < 0.05)
                        Performance has improved.
```

Closes #892
2025-02-05 10:46:04 +02:00
Pekka Enberg
0b0681c9f8 core/vdbe: Lazy cursor borrowing
This saves a few more nanoseconds:

```
Execute `SELECT 1`/Limbo
                        time:   [44.964 ns 45.064 ns 45.160 ns]
                        change: [-14.371% -13.724% -13.214%] (p = 0.00 < 0.05)
                        Performance has improved.
```
2025-02-05 09:47:17 +02:00
Pekka Enberg
e3169e5dcd Merge 'Simplify Statement::step()' from Pekka Enberg
...brings back a few nanoseconds.

Closes #891
2025-02-05 09:26:04 +02:00
Pekka Enberg
23cd8b10c3 core: Unify StepResult structs
...also simplify Statement::step() to get some performance back.

Before:

```
Execute `SELECT 1`/Limbo
                        time:   [49.128 ns 50.425 ns 52.604 ns]
```

After:

```
Execute `SELECT 1`/Limbo
                        time:   [49.128 ns 50.425 ns 52.604 ns]
```
2025-02-05 09:09:32 +02:00
Pekka Enberg
7573fc62e6 core: Unify Row and Record structs
They're exactly the same thing.
2025-02-05 09:04:52 +02:00
Glauber Costa
ac6dc9f9e8 clarify README and future plans 2025-02-04 19:44:04 -05:00
Pekka Enberg
20ffcb9d48 Merge 'core: Switch to parking_lot for RwLock' from Pekka Enberg
We really need to make the WAL lock less expensive, but switching to
`parking_lot` is anyway something we should do.
Before:
```
Execute `SELECT 1`/Limbo
                        time:   [56.230 ns 56.463 ns 56.688 ns]
```
After:
```
Execute `SELECT 1`/Limbo
                        time:   [52.003 ns 52.132 ns 52.287 ns]
```

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #887
2025-02-04 20:55:54 +02:00
Pekka Enberg
1aca47a384 Replace cover art with Turso branding
I socially engineerd Jamie into generating a Turso branded cover image
by merging something he hated, which was AI-generated image of a dark
forest. Thanks Jamie!
2025-02-04 20:47:33 +02:00
Marcus Nilsson
01492cf46f add support for json_set
Test cases are included.
Related to #127
2025-02-04 19:09:58 +01:00
Pekka Enberg
f004528e35 Update README.md 2025-02-04 20:07:21 +02:00
Pekka Enberg
68606d4d8c Update README.md cover art 2025-02-04 20:03:18 +02:00
Pekka Enberg
541270e38f Update README.md 2025-02-04 19:56:47 +02:00
Pekka Enberg
002ffb2e4d Update README.md 2025-02-04 19:35:07 +02:00
Pekka Enberg
4ad865af1d Merge 'Add Go example to main repo README' from Preston Thorpe
Now that we have a release that includes the Go bindings, we can have an
example where `go get github.com/tursodatabase/limbo` is used 👍

Closes #888
2025-02-04 19:31:47 +02:00
PThorpe92
1cbd33499e Add Go example to README 2025-02-04 11:53:11 -05:00
Marcus Nilsson
3478352b18 move extraction of JsonPath from OwnedValue to separate function 2025-02-04 17:49:49 +01:00
Pekka Enberg
e4d7474372 core: Switch to parking_lot for RwLock
We really need to make the WAL lock less expensive, but switching to
`parking_lot` is anyway something we should do.

Before:

```
Execute `SELECT 1`/Limbo
                        time:   [56.230 ns 56.463 ns 56.688 ns]
```

After:

```
Execute `SELECT 1`/Limbo
                        time:   [52.003 ns 52.132 ns 52.287 ns]
```
2025-02-04 18:38:33 +02:00
Pekka Enberg
750164fb85 Merge 'Sqlean Time extension' from Pedro Muniz
This PR implements a sqlean time compatible extension. I would
appreciate some help to review my code and see if there are ways to
enhance it. Also, if there is some edge case, I have missed please tell
me.
https://github.com/nalgeon/sqlean/blob/main/docs/time.md

Closes #854
2025-02-04 18:27:44 +02:00
Pekka Enberg
733f7de688 Merge branch 'main' into feature/time-ext 2025-02-04 18:27:14 +02:00
Pekka Enberg
f69804969c Merge 'Adding checkpoint result' from Sonny
### What?
adding checkpoint result returning number of pages in wal and num pages
checkpointed.
Part of #696
### Context
SQLite returns in checkpoint result of calling `pragma wal_checkpoint;`
`0|3|3` while limbo returns `0|0|0`.
https://sqlite.org/pragma.html#pragma_wal_checkpoint
- 1st col: 1 (checkpoint SQLITE_BUSY) or 0 (not busy).
- 2nd col: # modified pages written to wal file
- 3rd col: # pages moved to db after checkpoint
This PR aims to add 2nd and 3rd column to the checkpoint result.
SQLite
```
sqlite3 test.db
sqlite> pragma journal_mode=wal;
wal
sqlite> pragma journal_mode;
wal
sqlite> create table t1 (id text);
sqlite> insert into t1(id) values (1),(2);
sqlite> select * from t1;
1
2
sqlite> pragma wal_checkpoint;
0|3|3
```
Limbo
```
./target/debug/limbo test.db
Limbo v0.0.13
Enter ".help" for usage hints.
limbo> pragma journal_mode;
wal
limbo> create table t1(id text);
limbo> insert into t1(id) values (1),(2);
limbo> select * from t1;
1
2
# current the 2nd and 3rd columns are hard coded in limbo to 0
limbo> pragma wal_checkpoint;
0|0|0
```

Closes #827
2025-02-04 18:26:24 +02:00
Pekka Enberg
a045866a6c Merge 'core: Fix benchmark panic if syscall is interrupted' from Pekka Enberg
Fixes the following panics:
Benchmarking Execute `SELECT * FROM users LIMIT ?`/Limbo/100: Profiling
for 5.0000 sthread 'main' panicked at core/benches/benchmark.rs:69:43:
called `Result::unwrap()` on an `Err` value: IOError(Os { code: 4, kind:
Interrupted, message: "Interrupted system call" })
note: run with `RUST_BACKTRACE=1` environment variable to display a
backtrace

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #882
2025-02-04 18:25:34 +02:00
Pekka Enberg
b20ad42892 Merge 'github: Fix Python release workflow' from Pekka Enberg
We need to switch to `download-artifact@v4` because v3 is dead...

Closes #883
2025-02-04 18:25:26 +02:00
Pekka Enberg
bf1ef13c91 Merge 'Add Printf Support' from Zaid Humayun
Add basic printf function support in limbo
![Screenshot 2025-02-04 at 8 08 23 PM](https://github.com/user-
attachments/assets/b12931eb-8e79-4c8a-af77-c25c34cc5834)

Closes #886
2025-02-04 17:53:27 +02:00
Zaid Humayun
15b46e73eb test fix 2025-02-04 21:02:51 +05:30
Zaid Humayun
39d57ba541 test fix again 2025-02-04 20:58:03 +05:30
Zaid Humayun
4cb6f8cc6a fixing: failing test was expecting 2.50000
this test is behaving differently in different environments
2025-02-04 20:53:10 +05:30
Zaid Humayun
6a863b3da9 printf: this commit adds support for https://github.com/tursodatabase/limbo/issues/885 tracking printf functionality
this commit introduces basic support for printf functionality and doesn't include advanced modifiers like width etc.
2025-02-04 20:05:14 +05:30
Pekka Enberg
1817b75d9b github: Fix Python release workflow
We need to switch to `download-artifact@v4` because v3 is dead...
2025-02-04 15:55:19 +02:00
Pekka Enberg
73a345aa4d Limbo 0.0.14 2025-02-04 15:13:18 +02:00
Pekka Enberg
31552f0862 core/json: Kill some unwrap() calls 2025-02-04 14:55:19 +02:00
Pekka Enberg
2d9a54ccba Merge 'implement json_pretty' from Pedro Muniz
This PR implements json_pretty. At the moment, support for jsonb is
being added, so this function suffers from the same limitations as in
json(x). Also, I have not found a way to implement the same conversion
of Blob -> String that SQLite does. From my own experimentation, I
believe SQLite converts blobs to a lossy ascii representation, but I
would appreciate some help on this.

Closes #860
2025-02-04 14:51:47 +02:00