Commit Graph

380 Commits

Author SHA1 Message Date
Pekka Enberg
b5f5e40986 Merge 'prepare perf: dont eagerly allocate result column name strings' from Jussi Saurio
- Remove eagerly allocated `name` from `ResultSetColumn`
- `ResultSetColumn` can calculate `name()` on demand:
    - if it has an alias (`foo as bar`), use that
    - if it is a column reference, use that
    - otherwise return none, and callers can assign it a placeholder
name (like `column_1`)
- move the `plan.result_columns` and `plan.table_references` to
`Program` after preparing statement is done, so that column names can be
returned upon request
- make `name` in `Column` optional, not needed for pseudo tables and
sorters so avoids an extra string allocation
```sql
Prepare `SELECT 1`/Limbo/SELECT 1
                        time:   [756.80 ns 758.27 ns 760.04 ns]
                        change: [-3.3257% -3.0252% -2.7035%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  2 (2.00%) low severe
  3 (3.00%) low mild
  1 (1.00%) high mild
  2 (2.00%) high severe

Prepare `SELECT * FROM users LIMIT 1`/Limbo/SELECT * FROM users LIMIT 1
                        time:   [1.4646 µs 1.4669 µs 1.4696 µs]
                        change: [-6.4769% -6.2021% -5.9137%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) low severe
  3 (3.00%) low mild
  3 (3.00%) high severe

Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou...`
                        time:   [3.7256 µs 3.7311 µs 3.7376 µs]
                        change: [-4.5195% -4.2192% -3.9309%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) low severe
  2 (2.00%) low mild
  2 (2.00%) high mild
```

Closes #898
2025-02-05 18:20:01 +02:00
Jussi Saurio
795576b2ec dont eagerly allocate result column name strings 2025-02-05 17:53:23 +02:00
Pekka Enberg
fad479ac59 core/mvcc: Move source code to module 2025-02-05 13:25:16 +02:00
Pekka Enberg
9fdf54de2b Merge 'Small perf optimizations to statement preparation' from Jussi Saurio
```bash
Prepare `SELECT 1`/Limbo/SELECT 1
                        time:   [765.94 ns 768.26 ns 771.03 ns]
                        change: [-7.8340% -7.4887% -7.1406%] (p = 0.00 < 0.05)
                        Performance has improved.

Prepare `SELECT * FROM users LIMIT 1`/Limbo/SELECT * FROM users LIMIT 1
                        time:   [1.5673 µs 1.5699 µs 1.5731 µs]
                        change: [-10.810% -9.7122% -8.4951%] (p = 0.00 < 0.05)
                        Performance has improved.

Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou...
                        time:   [4.1331 µs 4.1421 µs 4.1513 µs]
                        change: [-9.3157% -9.0255% -8.7372%] (p = 0.00 < 0.05)
                        Performance has improved.
```
flamegraph for prepare `SELECT 1`:
<img width="1718" alt="Screenshot 2025-02-03 at 10 34 14"
src="https://github.com/user-
attachments/assets/ba67fe2f-78b2-4796-9a09-837d8e79fe62" />

Closes #872
2025-02-05 10:46:57 +02:00
Pekka Enberg
23cd8b10c3 core: Unify StepResult structs
...also simplify Statement::step() to get some performance back.

Before:

```
Execute `SELECT 1`/Limbo
                        time:   [49.128 ns 50.425 ns 52.604 ns]
```

After:

```
Execute `SELECT 1`/Limbo
                        time:   [49.128 ns 50.425 ns 52.604 ns]
```
2025-02-05 09:09:32 +02:00
Pekka Enberg
7573fc62e6 core: Unify Row and Record structs
They're exactly the same thing.
2025-02-05 09:04:52 +02:00
Pekka Enberg
e4d7474372 core: Switch to parking_lot for RwLock
We really need to make the WAL lock less expensive, but switching to
`parking_lot` is anyway something we should do.

Before:

```
Execute `SELECT 1`/Limbo
                        time:   [56.230 ns 56.463 ns 56.688 ns]
```

After:

```
Execute `SELECT 1`/Limbo
                        time:   [52.003 ns 52.132 ns 52.287 ns]
```
2025-02-04 18:38:33 +02:00
Pekka Enberg
f69804969c Merge 'Adding checkpoint result' from Sonny
### What?
adding checkpoint result returning number of pages in wal and num pages
checkpointed.
Part of #696
### Context
SQLite returns in checkpoint result of calling `pragma wal_checkpoint;`
`0|3|3` while limbo returns `0|0|0`.
https://sqlite.org/pragma.html#pragma_wal_checkpoint
- 1st col: 1 (checkpoint SQLITE_BUSY) or 0 (not busy).
- 2nd col: # modified pages written to wal file
- 3rd col: # pages moved to db after checkpoint
This PR aims to add 2nd and 3rd column to the checkpoint result.
SQLite
```
sqlite3 test.db
sqlite> pragma journal_mode=wal;
wal
sqlite> pragma journal_mode;
wal
sqlite> create table t1 (id text);
sqlite> insert into t1(id) values (1),(2);
sqlite> select * from t1;
1
2
sqlite> pragma wal_checkpoint;
0|3|3
```
Limbo
```
./target/debug/limbo test.db
Limbo v0.0.13
Enter ".help" for usage hints.
limbo> pragma journal_mode;
wal
limbo> create table t1(id text);
limbo> insert into t1(id) values (1),(2);
limbo> select * from t1;
1
2
# current the 2nd and 3rd columns are hard coded in limbo to 0
limbo> pragma wal_checkpoint;
0|0|0
```

Closes #827
2025-02-04 18:26:24 +02:00
Pekka Enberg
097e56c19f core: Kill Statement::query() method
It's a pointless wrapper on top of `step()` that introduce additional
memory allocation and deallocation.
2025-02-04 13:46:11 +02:00
Pekka Enberg
fb0a560d76 Merge 'dont store insn comments unless the query is EXPLAIN' from Jussi Saurio
We spend a lot of time especially in `GROUP BY` queries providing
helpful comments for `EXPLAIN`, even when the query is not an `EXPLAIN`.
So let's not do that
Closes #784
```sql
Prepare `SELECT first_name, count(1) FROM users GROUP BY first_name HAVING count(1) > 1 ORDER BY cou...
                        time:   [4.2724 µs 4.2783 µs 4.2848 µs]
                        change: [-6.1063% -5.7376% -5.3626%] (p = 0.00 < 0.05)
                        Performance has improved.
```
doesn't affect the other trivial prepare benchmarks

Closes #875
2025-02-04 13:10:37 +02:00
Jussi Saurio
1f888fea4f Dont fsync the WAL on read queries 2025-02-03 20:42:50 +02:00
Jussi Saurio
d182ddf514 dont store insn comments unless the query is EXPLAIN 2025-02-03 19:53:33 +02:00
sonhmai
022a8d7a83 core: return checkpoint result 2025-02-03 19:02:16 +07:00
sonhmai
1c803aa079 feat: add wal checkpoint result 2025-02-03 18:57:22 +07:00
Jussi Saurio
40f536fabb Dont store available_indexes on plan; only used in optimize_plan() 2025-02-03 12:52:14 +02:00
Nikita Sivukhin
2c958d7e2d derive Debug trait for limbo step result 2025-02-02 14:11:41 +04:00
Pekka Enberg
cfc585813b Merge 'implement sqlite_source_id function' from Glauber Costa
Closes #811
2025-01-29 09:45:00 +02:00
Glauber Costa
8f24d18ad8 implement sqlite_source_id function 2025-01-28 14:55:38 -05:00
Glauber Costa
bf1cfe3a1d avoid potentially expensive operations on prepare, query, execute
This simple patch makes sure we can operate with a reference to the
string instead of being forced to transform it to a string, and makes
sure that the Arc doesn't have to be cloned (which can be expensive in
multi-core systems).

This doesn't really make a large difference in benchmarks, given how
expensive Parse::new() is.
2025-01-28 13:44:34 -05:00
Pekka Enberg
e8600fa2a1 Merge branch 'main' into static 2025-01-27 09:49:34 +02:00
Pekka Enberg
7967cc5efc core: Kill Rows wrapper struct
It's just an useless wrapper, kill it.
2025-01-26 16:27:19 +02:00
Jussi Saurio
faa6d0c69d use vec for cursors, not map 2025-01-26 14:04:05 +02:00
Pekka Enberg
8942c38bda core: Fix Statement::reset()
The first rule of writing fast programs: don't use dynamic memory
allocation!

Brings back some performance for the `SELECT 1` micro-benchmark,
although we're still not where we need to be.
2025-01-26 11:47:28 +02:00
PThorpe92
32c985f9a8 Progress on Go bindings, add prepare + query statement 2025-01-25 23:01:46 -05:00
PThorpe92
545990f806 Support returning column names from prepared statement 2025-01-23 11:02:31 -05:00
PThorpe92
f13d035965 Enable wasm to static link extensions 2025-01-21 09:36:49 -05:00
PThorpe92
cc63aac305 Fix tests to account for built-in extensions 2025-01-21 09:32:44 -05:00
PThorpe92
3d188eba0f Enable staticly linking with builtin extensions 2025-01-21 09:32:43 -05:00
Krishna Vishal
04fd5a40d6 Finalize the parser in the case of Error while running queries. This resets the parser stack and prevents triggering the assertion and thereby panic.
Closes https://github.com/tursodatabase/limbo/issues/742
2025-01-20 16:10:35 +05:30
ben594
28ce68091f Modified changes and total_changes scalarfuncs to be more like sqlite
Fmt and clippy

Remove print
2025-01-19 20:51:13 -05:00
Pekka Enberg
f5e5428d45 Merge 'Syntactic improvements' from Jorge López Tello
This is a purely syntactic PR. It doesn't change behavior, just rewrites
some loops and removes unneeded parts, like lifetime annotations and
references. Mainly because the Clippy and IDE warnings get annoying.
Don't worry about the number of commits, I just separated based on type
of change.

Closes #732
2025-01-19 12:17:28 +02:00
Pekka Enberg
5851d6643a Revert "core: Move re-exports at top of lib.rs"
This reverts commit c8a979eb4b. It
conflicts with a cleanup PR I am about to merge.
2025-01-19 12:17:06 +02:00
Pekka Enberg
c8a979eb4b core: Move re-exports at top of lib.rs
Clean up the code a bit by moving re-exports at the top of lib.rs to
make them more visible to the reader.
2025-01-19 11:34:59 +02:00
PThorpe92
956320b7d0 Fix scalar API in extensions, add some error handling 2025-01-18 15:19:35 -05:00
Jorge López
e4ab2fb273 syntactic changes: rewrite loop with while 2025-01-18 19:19:49 +01:00
Jorge López
f160206cdd syntactic changes: replace unwrap() with ? in functions that return Result<...> 2025-01-18 18:33:43 +01:00
Jorge López
86a4714711 syntactic changes: remove unneeded paths when the type is already imported 2025-01-18 18:29:12 +01:00
PThorpe92
0c737d88f7 Support aggregate functions in Extensions 2025-01-17 14:13:57 -05:00
Pekka Enberg
9209641a07 cargo fmt 2025-01-16 14:43:23 +02:00
Pekka Enberg
93903555aa Rename limbo_extension crate to limbo_ext 2025-01-16 14:40:52 +02:00
Levy A.
9b8722f38e refactor: more well rounded implementation
`?0` parameters are now handled by the parser.
2025-01-15 16:53:26 -03:00
Levy A.
5de2694834 feat: more parameter support
add `Statement::{parameter_index, parameter_name, parameter_count,
bind_at}`. some refactoring is still needed, this is quite a rough
iteration
2025-01-15 16:51:04 -03:00
Levy A.
d3582a382f fix: small bugs 2025-01-15 16:51:04 -03:00
Levy A.
6e0ce3dd01 chore: cargo fmt 2025-01-15 16:51:04 -03:00
Levy A.
08c8c655e9 feat: initial implementation of Statement::bind 2025-01-15 16:51:04 -03:00
psvri
9cc9577c91 Run all statements from sql argument in cli 2025-01-15 18:19:39 +05:30
PThorpe92
23d9d09b70 Add load_extension function, resolve shared lib extensions 2025-01-14 12:01:07 -05:00
PThorpe92
9c208dc866 Add tests for first extension 2025-01-14 07:27:35 -05:00
PThorpe92
3099e5c9ba Improve api, standardize conversions between types, finish extension 2025-01-14 07:20:50 -05:00
PThorpe92
3412a3d4c2 Rough design for extension api/draft extension 2025-01-14 07:20:48 -05:00