Commit Graph

3074 Commits

Author SHA1 Message Date
pedrocarlo
738e2cc06c do not emit ephemeral plan when doing a SeekRowId + emit Delete instruction when rowid in set clause 2025-06-29 17:12:24 -03:00
Pekka Enberg
d377f4c948 Move completion extension dependency to CLI
We never need it in core anyway.
2025-06-29 13:32:17 +03:00
Pekka Enberg
725c3e4ddc Rename limbo_sqlite3_parser crate to turso_sqlite3_parser 2025-06-29 12:34:46 +03:00
Pekka Enberg
eb0de4066b Rename limbo_ext crate to turso_ext 2025-06-29 12:14:08 +03:00
Pekka Enberg
51b6e347a8 Merge 'core: Add Antithesis-aware turso_assert' from Pekka Enberg
This adds a `turso_assert` macro that is Antithesis aware when
`antithesis` feature flag is enabled. I did not yet convert any call-
sites to use it.

Closes #1880
2025-06-29 12:11:44 +03:00
Pekka Enberg
eec994386b Rename limbo_macros to turso_macros 2025-06-29 12:00:17 +03:00
Pekka Enberg
645c0bd796 core: Add Antithesis-aware turso_assert
This adds a `turso_assert` macro that is Antithesis aware when
`antithesis` feature flag is enabled. I did not yet convert any
call-sites to use it.

Co-authored-by: Nikita Sivukhin <sivukhin@turso.tech>
2025-06-29 11:46:18 +03:00
Pekka Enberg
53ba3ff926 Rename limbo_core crate to turso_core 2025-06-29 09:59:17 +03:00
Pekka Enberg
a60da8186f Merge 'Remove dependency on test extension pkg' from Preston Thorpe
To prevent having to publish the crate on crates.io

Closes #1865
2025-06-27 18:22:45 +03:00
PThorpe92
708aaf95ee Add generate_series to default built-in extensions 2025-06-27 09:48:17 -04:00
PThorpe92
0e26cf77cb Remove dependency on testing extension crate 2025-06-27 09:41:18 -04:00
Pekka Enberg
5791ab9dff Merge 'Cache reserved_space and page_size values at Pager init to prevent doing redundant IO' from Krishna Vishal
### Problem
Profiling revealed that `usable_space()` calls were consuming 60% of
total execution time for simple SELECT queries, making Limbo
approximately `6x` slower than SQLite for SELECT operations.
The bottleneck was caused by `usable_space()` performing expensive I/O
operations on every call to read `page_size` and `reserved_space` from
the database header, despite `page_size` values being effectively
immutable after database initialization. Only `reserved_space` is
allowed to increase in SQLite.
Evidence: https://share.firefox.dev/44tCUIy
### Solution
Implemented OnceCell-based caching for both page_size and reserved_space
values in the Pager struct:
`page_size: OnceCell<u16>` - Page size is immutable after database
initialization per SQLite specification
`reserved_space: OnceCell<u8>` - Reserved space rarely changes and only
grows, safe to cache
### Performance Impact
Benchmark results: Simple SELECT query time reduced from ~2.89ms to
~1.29ms (~55% improvement)

Closes #1852
2025-06-27 16:40:14 +03:00
Pere Diaz Bou
22cb95bc5d execute run on multiple parsed statements 2025-06-27 14:02:47 +02:00
Pekka Enberg
09ba89e2ba core/translate: Replace todo with bail_parse_error
No point in crashing the whole app if someone attempts to change page
size.
2025-06-27 13:42:49 +03:00
Pekka Enberg
c12b291f9a Merge 'Fix evaluation of ISNULL/NOTNULL in OR expressions' from Piotr Rżysko
Previously, the `jump_if_condition_is_true` flag was not respected. As a
result, for expressions like <`ISNULL`/`NOTNULL`> `OR` <rhs>, the <rhs>
expression was evaluated even when the left-hand side was true, and its
value was incorrectly used as the final result.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1846
2025-06-27 13:15:21 +03:00
Pere Diaz Bou
8e0f8041ed properly set database header contents on initialization
After moving page1 write to be async I moved the contents update to
wrong place. This should fix it.
2025-06-27 11:44:11 +02:00
Krishna Vishal
cda1ab8d76 Use OnceCell instead of OnceLock. 2025-06-27 13:32:03 +05:30
Krishna Vishal
af2ab87810 Cache reserved_space and page_size values at Pager init.
We use `OnceLock` for this. TODO: Invalidate reserved_space when
we make functionality the to change it.
2025-06-27 12:51:11 +05:30
Piotr Rzysko
116df2ec86 Fix evaluation of ISNULL/NOTNULL in OR expressions
Previously, the `jump_if_condition_is_true` flag was not respected. As a
result, for expressions like <`ISNULL`/`NOTNULL`> `OR` <rhs>, the <rhs>
expression was evaluated even when the left-hand side was true, and its
value was incorrectly used as the final result.
2025-06-27 08:21:40 +02:00
Pekka Enberg
e1c8c676ca Merge 'Support insersect operator for compound select' from meteorgan
Closes: #1575

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1793
2025-06-27 09:13:16 +03:00
meteorgan
0ed94f13f5 resolve conflicts 2025-06-27 11:50:19 +08:00
meteorgan
51764d882e fix comments 2025-06-27 11:50:19 +08:00
meteorgan
2c4847210f ajust code to accommodate index_experimental feature 2025-06-27 11:50:19 +08:00
meteorgan
41def8895f make intersect in compound work with insert 2025-06-27 11:50:19 +08:00
meteorgan
1fcc2ddd90 support limit 2025-06-27 11:50:19 +08:00
meteorgan
d4789d0a05 add tests 2025-06-27 11:50:19 +08:00
meteorgan
cd36fc26fd support intersect operation for compound select 2025-06-27 11:50:19 +08:00
pedrocarlo
1dc28e32f0 fix io_uring completion + clippy 2025-06-26 22:17:28 -03:00
pedrocarlo
bac5e4b563 refactor File and Database Storage to remove Arc<Connection> and return Arc<Connection> for caller to wait for completion 2025-06-26 22:17:28 -03:00
pedrocarlo
64d9193e7b refactor Completion to have a type field and lift common is_complete property 2025-06-26 22:17:27 -03:00
Pekka Enberg
572c722390 Merge 'write page1 on database initialization' from Pere Diaz Bou
Page 1 must be initialized and written as soon as possible without
marking page as dirty.
OpenEphemeral now requires a state machine to accomodate new
begin_write_tx semantics.

Closes #1839
2025-06-26 20:43:40 +03:00
Pere Diaz Bou
aa93b70a96 empty -> unitialized 2025-06-26 17:59:23 +02:00
Pere Diaz Bou
e341b80051 clippy 2025-06-26 15:01:54 +02:00
Pere Diaz Bou
4d80b8237d write page1 on database initialization
Page 1 must be initialized and written as soon as possible without
marking page as dirty.
2025-06-26 14:44:23 +02:00
Pekka Enberg
2fc5c0ce5c Switch to runtime flag for enabling indexes
Makes it easier to test the feature:

```
$ cargo run --  --experimental-indexes
Limbo v0.0.22
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database
limbo> CREATE TABLE t(x);
limbo> CREATE INDEX t_idx ON t(x);
limbo> DROP INDEX t_idx;
```
2025-06-26 10:07:28 +03:00
Pekka Enberg
a48198ec60 Merge 'Rollback simple support' from Pere Diaz Bou
Support for simple interactive rollback like:
```sql
    create table t (x);
    insert into t values (1);
    begin;
    insert into t values (2);
    rollback;
    select * from t;
```
This PR also fixes some other issues I found while debugging:
* Checkpoint would never `clear_dirty` on pages in page cache.
* Auto commit for interactive transactions was not respected so any
`insert` after `begin` would flush frames regardless of `auto_commit`
state.
* `max_frame` on wal shared state was being updated after every
`append_frame` which was incorrect, as another transaction would be able
to use that new `max_frame` even tho the transaction could've rolled
back. Instead we update the private copy of `max_frame` and only update
it at the end.
Follow up for later are savepoints which require implementing a
subjournal to track savepoints and their modified pages.

Closes #1825
2025-06-25 20:02:09 +03:00
Pere Diaz Bou
c02337c8cc clear dirty pages on rollback 2025-06-25 14:01:53 +02:00
Pere Diaz Bou
34a6d236ab fix comp 2025-06-25 14:01:53 +02:00
Pere Diaz Bou
96c30be488 more clippy 2025-06-25 14:01:53 +02:00
Pere Diaz Bou
0119b0f99d clippy 2025-06-25 14:01:53 +02:00
Pere Diaz Bou
a3ad138df8 checkpoint clear dirty page if it was on cache 2025-06-25 14:01:53 +02:00
Pere Diaz Bou
22f9cd695d commit_txn track rollback case 2025-06-25 14:00:57 +02:00
Pere Diaz Bou
b6a948e513 add cute log for translate input 2025-06-25 14:00:57 +02:00
Pere Diaz Bou
8517cea530 add finish append frames log 2025-06-25 14:00:57 +02:00
Pere Diaz Bou
bdd2010df3 autocommit rollback 2025-06-25 14:00:57 +02:00
Jussi Saurio
cb29f325e2 No need for Database::is_empty to be pub 2025-06-25 14:53:39 +03:00
Pere Diaz Bou
d66c683a4c implement rollback translation 2025-06-25 13:45:32 +02:00
Jussi Saurio
27b3ecf599 core/db&pager: fix locking for initializing empty database
When `struct Database` is constructed, store `is_empty` as an
`Arc<AtomicBool>` - the value is true if:

1. DB size is zero
2. WAL has no frames

When `struct Pager` is constructed, this `Arc` is simply cloned.
When any connection runs a transaction it will first check `is_empty`,
and if the DB is empty, it will lock `init_lock` and then check `is_empty`
again, and if it's still true, it allocates page1 and stores `false` in
the `is_empty` `AtomicBool` and drops the lock.

---

Note that Limbo can currently have a zero DB and a WAL with frames,
as we have no special logic for folding page1 to the main DB file
during initialization.

Page 1 allocation currently happens on the first transaction (read or
write, due to having to support `select * from sqlite_schema` on an
empty DB; we should really check how SQLite actually does this.).
2025-06-25 14:45:21 +03:00
Jussi Saurio
480f0a04b5 make clippy happy about mutating database_size immediately after default construction 2025-06-24 14:41:50 -03:00
Jussi Saurio
f21cde9501 post-rebase fixes 2025-06-24 14:41:50 -03:00