Commit Graph

2753 Commits

Author SHA1 Message Date
Pekka Enberg
9e65d7ddea Merge 'perf/prepare: box even more stuff to shrink the YYMINORTYPE enum further' from Jussi Saurio
the boxings will continue until morale improves
<img width="621" alt="Screenshot 2025-02-09 at 14 16 21"
src="https://github.com/user-
attachments/assets/887d218d-3db7-4a64-ad81-b7431adb23bb" />

Closes #947
2025-02-09 17:36:03 +02:00
Jussi Saurio
a8685c8086 sqlite3-parser: box the Expr in Vacuum 2025-02-09 14:24:55 +02:00
Jussi Saurio
9b0997a60d sqlite3-parser: separate boxed CreateVirtualTable struct 2025-02-09 14:24:55 +02:00
Jussi Saurio
36a3cb1d5e sqlite3-parser: box AlterTable 2025-02-09 14:11:34 +02:00
Jussi Saurio
72a055e5fe sqlite3-parser: box Pragma 2025-02-09 13:10:52 +02:00
Jussi Saurio
23c4106433 sqlite3-parser: separate boxed Insert struct 2025-02-09 13:10:21 +02:00
Jussi Saurio
f0d7d82e1d sqlite3-parser: Box the Expr in Detach 2025-02-09 13:10:21 +02:00
Jussi Saurio
32887518ce sqlite3-parser: separate boxed Delete struct 2025-02-09 13:10:21 +02:00
Jussi Saurio
40a8dc14cd sqlite3-parser: separate boxed SelectInner struct 2025-02-09 12:54:30 +02:00
Jussi Saurio
f75aca67bb sqlite3-parser: separate boxed TriggerCmd struct variants 2025-02-09 12:53:12 +02:00
Jussi Saurio
af920a317c sqlite3-parser: separate boxed Update struct 2025-02-09 12:53:12 +02:00
Jussi Saurio
575b484740 sqlite3-parser: separate boxed CreateTrigger struct 2025-02-09 12:50:00 +02:00
Jussi Saurio
358fda2ec7 sqlite3-parser: box the create table body 2025-02-09 12:42:53 +02:00
Jussi Saurio
d177f6195b sqlite3-parser: box big members of createindex 2025-02-09 12:34:53 +02:00
Pekka Enberg
e406a030e6 Merge 'Rework io_uring feature' from Jorge López Tello
This makes io_uring the default in CLI, but makes it non-default in
core. Before, if one built CLI without io_uring, core still built with
it as it was a default feature. To accommodate for the change, all
bindings have been updated to select the feature, except for WASM which
has a separate fs implementation.
This also adds some #[cfg] and #[allow] to silence unused-* warnings,
which I discovered when testing with different features disabled.

Closes #942
2025-02-09 08:46:29 +02:00
Pekka Enberg
3178e975b9 Merge 'perf/prepare: box many less frequently used AST nodes' from Jussi Saurio
`sqlite3-parser` spends a lot of time in `yy_reduce` assigning different
`enum YYMINORTYPE` members, and it suffers from bad performance because
the stack size of the enum is very large, and the size of individual
members varies wildly. This PR reduces the `YYMINORTYPE` stack size from
496 to 264 bytes and has the following effect:
Preparing statement:
```sql
Prepare `SELECT 1`/Limbo/SELECT 1
                        time:   [620.37 ns 621.08 ns 621.79 ns]
                        change: [-17.811% -17.582% -17.380%] (p = 0.00 < 0.05)
                        Performance has improved.

Prepare `SELECT * FROM users LIMIT 1`/Limbo/SELECT * FROM users LIMIT 1
                        time:   [1.2215 µs 1.2231 µs 1.2248 µs]
                        change: [-12.926% -12.627% -12.272%] (p = 0.00 < 0.05)
                        Performance has improved.

Benchmarking Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou...: Collecting 100 samples in estimated 5.0152 s (
Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou...
                        time:   [3.1056 µs 3.1096 µs 3.1138 µs]
                        change: [-13.279% -12.995% -12.712%] (p = 0.00 < 0.05)
                        Performance has improved.

```
Execute (mainly to check for regressions):
```sql
Execute `SELECT * FROM users LIMIT ?`/Limbo/1
                        time:   [402.19 ns 402.75 ns 403.36 ns]
                        change: [-3.4845% -2.4003% -1.6539%] (p = 0.00 < 0.05)
                        Performance has improved.

Execute `SELECT * FROM users LIMIT ?`/Limbo/10
                        time:   [2.7920 µs 2.7977 µs 2.8036 µs]
                        change: [-1.3135% -1.0123% -0.7132%] (p = 0.00 < 0.05)
                        Change within noise threshold.

Execute `SELECT * FROM users LIMIT ?`/Limbo/50
                        time:   [13.577 µs 13.633 µs 13.690 µs]
                        change: [-1.7709% -1.3575% -0.9563%] (p = 0.00 < 0.05)
                        Change within noise threshold.
```

Closes #936
2025-02-09 08:45:42 +02:00
Pekka Enberg
55e873cf49 Merge 'cli: Improve pretty mode table' from wyhaya
This PR replace `cli-table` with [comfy-
table](https://github.com/nukesor/comfy-table), with the following
improvements:
* Limit value to 1 line, regardless of whether they contain line breaks
* Truncate excess content when the complete content cannot be displayed
* Adapt to terminal width
* Fix #928
```sql
limbo> .mode pretty

limbo> CREATE TABLE "test" ("id" text NOT NULL, "age" integer, "height" real, PRIMARY KEY ("id"));
-- No output

limbo> INSERT INTO "test" ("id", "age", "height")
VALUES
('8f67b3fa-ad9a-4b51-947f-778b80a0b350', 477557, 79.34657),
('1ee52440-ccc7-4a88-bb25-46f2da1f9f69', 5963124, 43.86521),
('bb1f6009-dbcb-4e61-b958-b3a7f5153bba', 6822724, 16.318216);
-- No output

limbo> SELECT * FROM test;
┌──────────────────────────────────────┬─────────┬───────────┐
│ id                                   │ age     │ height    │
├──────────────────────────────────────┼─────────┼───────────┤
│ 8f67b3fa-ad9a-4b51-947f-778b80a0b350 │  477557 │  79.34657 │
├──────────────────────────────────────┼─────────┼───────────┤
│ 1ee52440-ccc7-4a88-bb25-46f2da1f9f69 │ 5963124 │  43.86521 │
├──────────────────────────────────────┼─────────┼───────────┤
│ bb1f6009-dbcb-4e61-b958-b3a7f5153bba │ 6822724 │ 16.318216 │
└──────────────────────────────────────┴─────────┴───────────┘
-- Column names are left-aligned and bolded
-- Text is left-aligned
-- Numbers are right-aligned

limbo> CREATE TABLE empty(id text, age integer, data blob);
limbo> SELECT * FROM empty;
┌────┬─────┬──────┐
│ id │ age │ data │
├────┼─────┼──────┤
└────┴─────┴──────┘
-- No rows

limbo> select * from test;
┌───────────────┬─────────┬───────────┐
│ id            │ age     │ height    │
├───────────────┼─────────┼───────────┤
│ 8f67b3fa-ad9… │  477557 │  79.34657 │
├───────────────┼─────────┼───────────┤
│ 1ee52440-ccc… │ 5963124 │  43.86521 │
├───────────────┼─────────┼───────────┤
│ bb1f6009-dbc… │ 6822724 │ 16.318216 │
└───────────────┴─────────┴───────────┘
-- Content will be truncated if the terminal width is insufficient
```
https://github.com/user-
attachments/assets/84dca32d-315f-4991-ad24-b2cdf11bc086

Closes #937
2025-02-09 08:45:26 +02:00
Pekka Enberg
7f9d3f7648 Merge 'simulator: fix shrinking bug' from Alperen Keleş
Fixes https://github.com/tursodatabase/limbo/issues/924

Closes #935
2025-02-09 08:45:11 +02:00
Pekka Enberg
ebbb84bb70 Merge 'github.com/penberg/limbo was moved to github.com/tursodatabase/limbo' from Gustavo Sverzut Barbieri
Adjust all the references since they were broken

Reviewed-by: Preston Thorpe (@PThorpe92)
Reviewed-by: Ricardo Dalarme (@ricardodalarme)

Closes #933
2025-02-09 08:44:39 +02:00
Jorge López
83b158fb3a core: silence some unused warnings when building without default features 2025-02-09 01:13:12 +01:00
Jorge López
c07c08aa98 core: make io_uring feature non-default. All crates that depend on core select it by default. This enables us to build CLI without io_uring, which before this commit would still have used io_uring in core. 2025-02-09 01:12:27 +01:00
Jorge López
62dea0b12b cli: select io_uring feature by default 2025-02-09 01:11:00 +01:00
Jorge López
be5ea350bb bindings: select io_uring feature from limbo_core explicitly as it will be made non-default 2025-02-09 01:10:35 +01:00
alpaylan
7ddbcf07af fix formatting 2025-02-08 15:41:57 -05:00
alpaylan
69d72da837 fix the diff computing algorithm 2025-02-08 12:53:25 -05:00
Jussi Saurio
4faadd86b0 sqlite3-parser: box the InsertBody 2025-02-08 18:10:26 +02:00
Jussi Saurio
781aa3b5d6 sqlite3-parser: box the having clause in GroupBy 2025-02-08 18:10:26 +02:00
Jussi Saurio
2a82091cb3 sqlite3-parser: box the where clause in Update 2025-02-08 18:10:26 +02:00
Jussi Saurio
7426204204 sqlite3-parser: box Following and Preceding in FrameBound 2025-02-08 18:10:26 +02:00
Jussi Saurio
ac7f9d67b7 sqlite3-parser: box large members of Upsert 2025-02-08 18:10:25 +02:00
Jussi Saurio
f341474fee sqlite3-parser: box large members of CreateTrigger 2025-02-08 18:10:25 +02:00
Jussi Saurio
0dba39b025 sqlite3-parser: box everything in Attach 2025-02-08 18:10:25 +02:00
Jussi Saurio
670dac5939 sqlite3-parser: box the where clause in Delete 2025-02-08 18:10:25 +02:00
wyhaya
e9046fef78 cli: Improve pretty mode table 2025-02-09 00:01:07 +08:00
alpaylan
3ae3e650ae fix watch mode bug deleting the last interaction of a property 2025-02-08 10:59:46 -05:00
alpaylan
4362bc16a3 fix formatting 2025-02-08 09:37:08 -05:00
alpaylan
53ce08aefa Merge branch 'main' of https://github.com/tursodatabase/limbo 2025-02-08 09:35:21 -05:00
alpaylan
6308ce4544 fix the shrinking file and poison errors 2025-02-08 09:34:51 -05:00
Gustavo Sverzut Barbieri
cd2d817c10 github.com/penberg/limbo was moved to github.com/tursodatabase/limbo
Adjust all the references since they were broken
2025-02-08 10:21:49 -03:00
Pekka Enberg
a1a9218131 Merge 'cli: Make pretty mode pretty like DuckDB' from Pekka Enberg
DuckDB is pretty, I want to be pretty!
```
limbo> CREATE TABLE t(x); INSERT INTO t VALUES (1), (2), (3);
limbo> .mode pretty
limbo> SELECT * FROM t;
┌───┐
│ x │
├───┤
│ 1 │
├───┤
│ 2 │
├───┤
│ 3 │
└───┘
```

Closes #931
2025-02-08 12:04:01 +02:00
Pekka Enberg
3deac98d40 cli: Make pretty mode pretty like DuckDB
DuckDB is pretty, I want to be pretty!

```
limbo> CREATE TABLE t(x); INSERT INTO t VALUES (1), (2), (3);
limbo> .mode pretty
limbo> SELECT * FROM t;
┌───┐
│ x │
├───┤
│ 1 │
├───┤
│ 2 │
├───┤
│ 3 │
└───┘
```
2025-02-08 11:39:21 +02:00
Pekka Enberg
24d65f9de2 Merge 'Fix Parser continuing to parse after encountering an error' from Krishna Vishal
Added  `had_true: bool` to `Parser` to track the state of error
encounter, this will be set to `true` once we encounter an error.
If `had_error` is set to true we early return `Ok(None)`.
`FallibleIterator` works in the following way [when `next()`
returns](https://docs.rs/fallible-iterator/latest/fallible_iterator/trai
t.FallibleIterator.html#tymethod.next):
- `Ok(Some(item))` if there's a valid next item.
- `Ok(None)` if the iteration is complete.
- `Err(e)` if there's an error.
I have also added a unit test to check the above contract holds when
parsing `SELECT FROM foo`.
After the fix:
```
limbo> SELECT FROM foo;

  × near "FROM": syntax error at (1, 12)
   ╭────
 1 │ SELECT FROM foo;
   ·           ▲
   ·           ╰── syntax error
   ╰────
```
Closes #865

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #879
2025-02-08 11:17:27 +02:00
Pekka Enberg
71d2a9b6ab Merge 'Replace hashmap for io_uring pending ops with static array' from Preston Thorpe
Since we are already round robin'ing our `iovecs` in the same way, and
the map will never be larger than MAX_IOVECS, we can remove the cost of
hashing and keep o(1) lookups.

Closes #922
2025-02-08 11:15:54 +02:00
Pekka Enberg
b17610ab77 core/mvcc: Minor code cleanups
Make the source file readable from top to bottom by moving private
functions at the end of the struct implementation.
2025-02-08 10:55:13 +02:00
Pekka Enberg
f9e0650e5e Merge 'MVCC: fix write conflict handling' from Jussi Saurio
Fixes three bugs:
- When a transaction commits, it should:
  1. set the end timestamp of the old version to its own **end
timestamp**,
  2. set the begin timestamp of the new version to its own **end
timestamp**
    * we were erroneously setting it to the transaction's begin
timestamp. see https://www.cs.cmu.edu/~15721-f24/papers/Hekaton.pdf page
299 diagram.
- When checking for write conflicts, the system should ignore versions
that the transaction cannot see. We were checking for write conflicts
before visibility, allowing a tx to always see the latest committed
version and to always overwrite it, since latest committed versions dont
have an end timestamp.
- When checking for write conflicts, the _presence_ of an end timestamp
should always indicate that it is not the latest version and thus a
newer committed version exists, forcing the transaction to abort.
see https://www.cs.cmu.edu/~15721-f24/papers/Hekaton.pdf page 301
section 2.6. Updating a version

Reviewed-by: Pekka Enberg <penberg@iki.fi>

Closes #927
2025-02-08 10:54:00 +02:00
Pekka Enberg
90e7f021fd Merge 'cli: Add column names in Pretty mode' from wyhaya
This is especially useful if you frequently run SQL query like `SELECT *
...`.
## Before
```
limbo> .mode pretty
limbo> select 123 as column1, 'Hello world' as column_2;
+-----+-------------+
| 123 | Hello world |
+-----+-------------+
```
## After
```
limbo> .mode pretty
limbo> select 123 as column1, 'Hello world' as column_2;
+---------+-------------+
| column1 | column_2    |
+---------+-------------+
| 123     | Hello world |
+---------+-------------+
```
---
In the future, if limbo adds a method like `.decl_type()`, we can also
display the data type.
###### DuckDB
```
D select 123 as column1, 'Hello world' as column_2;
┌─────────┬─────────────┐
│ column1 │  column_2   │
│  int32  │   varchar   │
├─────────┼─────────────┤
│   123   │ Hello world │
└─────────┴─────────────┘
```

Closes #925
2025-02-08 10:42:03 +02:00
Jussi Saurio
791255fd8c MVCC: Add a few comments 2025-02-08 10:20:48 +02:00
Jussi Saurio
fab105c10c MVCC: fix write conflict handling 2025-02-08 10:10:09 +02:00
Pekka Enberg
9657920dd7 Merge 'Add java section in README.md' from Kim Seon Woo
Added a section for java in README.md

Closes #923
2025-02-08 09:15:04 +02:00
wyhaya
13062a1479 cli: Add column names in Pretty mode 2025-02-08 15:11:09 +08:00