Commit Graph

156 Commits

Author SHA1 Message Date
Avinash Sajjanshetty
53f9c0dc7a Add support for lord AEGIS, the fastest and the greatest 2025-08-24 16:15:11 +05:30
Levy A.
186e2f5d8e switch to new parser 2025-08-21 15:19:16 -03:00
Avinash Sajjanshetty
40a209c000 simplify feature flag usage for encryption 2025-08-20 12:49:38 +05:30
Avinash Sajjanshetty
100a0d8e97 Add encryption module
Let's add an encryption module, hard coded to use AES 256 GCM.
Other required parameters are also hard coded and will be made
configurable in the future PRs.

The module is behind a `encryption` feature flag.
2025-08-20 11:38:11 +05:30
Jussi Saurio
87bf488bbc chore: use rusqlite 0.37 with bundled sqlite everywhere 2025-08-11 15:13:57 +03:00
Piotr Rzysko
40a8058083 Use mimalloc in benchmark.rs 2025-08-11 08:02:35 +02:00
pedrocarlo
edae65fb5f global allocator should not be set for library, only for executables 2025-08-07 13:41:50 -03:00
Nikita Sivukhin
0adb40534c hind dangerous methods behind conn_raw_api feature 2025-08-04 12:40:28 +04:00
Jussi Saurio
f619556344 Merge 'Direct DatabaseHeader reads and writes – with_header and with_header_mut' from Levy A.
This PR introduces two methods to pager. Very much inspired by
`with_schema` and `with_schema_mut`. `Pager::with_header` and
`Pager::with_header_mut` will give to the closure a shared and unique
reference respectively that are transmuted references from the `PageRef`
buffer.
This PR also adds type-safe wrappers for `Version`, `PageSize`,
`CacheSize` and `TextEncoding`, as they have special in-memory
representations.
Writing the `DatabaseHeader` is just a single `memcpy` now.
```rs
pub fn write_database_header(&self, header: &DatabaseHeader) {
    let buf = self.as_ptr();
    buf[0..DatabaseHeader::SIZE].copy_from_slice(bytemuck::bytes_of(header));
}
```
`HeaderRef` and `HeaderRefMut` are used in the `with_header*` methods,
but also can be used on its own when there are multiple reads and writes
to the header, where putting everything in a closure would add too much
nesting.

Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #2234
2025-07-31 10:02:47 +03:00
PThorpe92
7b2163208b batch backfilling pages when checkpointing 2025-07-30 19:42:48 -04:00
Levy A.
e35fdb8263 feat: zero-copy DatabaseHeader 2025-07-30 17:33:59 -03:00
Diego Reis
bab10909c3 Disable extension loading for wasm
We should enable it later when wasm become more mature
2025-07-28 14:49:07 -03:00
Iaroslav Zeigerman
78f3bf3475 Core: Introduce external sorting 2025-07-18 07:28:36 +02:00
Pekka Enberg
36b550cca4 Merge 'Fix boxed memory leaks' from Ihor Andrianov
We should recreate original box to drop it properly
Also made a fast path for hashing. When key div by 2. It should decrease
cpu cycles on hot path by x10 approximately
This thing is tricky, made a long running test that verify bug, put
#[ignore] on it to not slow down CI

Reviewed-by: Preston Thorpe (@PThorpe92)

Closes #1873
2025-07-02 19:42:54 +03:00
Ihor Andrianov
41a11afe7c leaking box memory 2025-07-01 17:27:47 +03:00
Pekka Enberg
60191e7c7b Move series extension to core
It's part of upstream SQLite too.
2025-06-30 10:29:34 +03:00
Pekka Enberg
c9945950e8 core: Remove dependencies to extensions
We don't want to publish all extensions on crates.io, at least not for now.
2025-06-30 10:01:03 +03:00
Pekka Enberg
39fd84f297 Move time extension to core 2025-06-30 10:01:03 +03:00
Pekka Enberg
12131babae Move UUID extension to core
We want to bundle the UUID extension by default so move the code to core.
2025-06-30 09:54:13 +03:00
Pekka Enberg
d377f4c948 Move completion extension dependency to CLI
We never need it in core anyway.
2025-06-29 13:32:17 +03:00
Pekka Enberg
725c3e4ddc Rename limbo_sqlite3_parser crate to turso_sqlite3_parser 2025-06-29 12:34:46 +03:00
Pekka Enberg
eb0de4066b Rename limbo_ext crate to turso_ext 2025-06-29 12:14:08 +03:00
Pekka Enberg
51b6e347a8 Merge 'core: Add Antithesis-aware turso_assert' from Pekka Enberg
This adds a `turso_assert` macro that is Antithesis aware when
`antithesis` feature flag is enabled. I did not yet convert any call-
sites to use it.

Closes #1880
2025-06-29 12:11:44 +03:00
Pekka Enberg
eec994386b Rename limbo_macros to turso_macros 2025-06-29 12:00:17 +03:00
Pekka Enberg
645c0bd796 core: Add Antithesis-aware turso_assert
This adds a `turso_assert` macro that is Antithesis aware when
`antithesis` feature flag is enabled. I did not yet convert any
call-sites to use it.

Co-authored-by: Nikita Sivukhin <sivukhin@turso.tech>
2025-06-29 11:46:18 +03:00
Pekka Enberg
53ba3ff926 Rename limbo_core crate to turso_core 2025-06-29 09:59:17 +03:00
PThorpe92
708aaf95ee Add generate_series to default built-in extensions 2025-06-27 09:48:17 -04:00
PThorpe92
0e26cf77cb Remove dependency on testing extension crate 2025-06-27 09:41:18 -04:00
Pekka Enberg
2fc5c0ce5c Switch to runtime flag for enabling indexes
Makes it easier to test the feature:

```
$ cargo run --  --experimental-indexes
Limbo v0.0.22
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database
limbo> CREATE TABLE t(x);
limbo> CREATE INDEX t_idx ON t(x);
limbo> DROP INDEX t_idx;
```
2025-06-26 10:07:28 +03:00
Jussi Saurio
133d498724 Implement a header_accessor module so that DatabaseHeader structs arent initialized on every access 2025-06-24 14:41:50 -03:00
Pere Diaz Bou
9ae4563bcd index_experimental flag to enable index usages
Currently indexes are the bulk of the problem with `UPDATE` and
`DELETE`, while we work on fixing those it makes sense to disable
indexing since they are not stable. We want to try to make everything
else stable before we continue with indexing.
2025-06-17 19:33:23 +02:00
pedrocarlo
4bbe780a34 add optional serde serialization and deserialization to limbo Value 2025-06-09 11:38:15 -03:00
Zaid Humayun
b9ae5e7ce9 removed omit_autovacuum from flags and fixed tests 2025-06-06 23:14:25 +05:30
Zaid Humayun
5827a33517 Beginnings of AUTOVACUUM
This commit introduces AUTOVACUUM to Limbo. It introduces the concept of ptrmap pages and also adds some additional instructions that are required to make AUTOVACUUM PRAGMA work
2025-06-06 23:14:22 +05:30
pedrocarlo
66f82e47f5 tpc-h with criterion and nyrkio 2025-05-23 01:32:16 -03:00
Piotr Rzysko
ad9d044a04 Add CSV extension 2025-05-21 09:22:59 +02:00
pedrocarlo
510c70e919 Create CollationSeq enum and functions. Move strum to workspace dependency to avoid version mismatch with Parser 2025-05-19 15:22:14 -03:00
Pere Diaz Bou
5eab588115 improve debug build validation speed
Various things:
* remove unnecessary debug_validate_cell calls
* Add SortedVec for keys in fuzz tests
* Validate btree's depth in fuzz test every 1K inserts to not overload
test with validations. We add `VALIDATE_BTREE`  env variable to enable
validation on every insert in case it is needed.
2025-05-19 09:53:15 +02:00
Jussi Saurio
4e05023bd3 Merge branch 'main' into ext-static-feature 2025-05-03 19:18:28 +03:00
Jussi Saurio
c9eb56b54a Merge 'Read only mode' from Pedro Muniz
Closes #1413 . Basically, SQLite emits a check in a transaction to see
if it is attempting to write. If the db is in read only mode, it throws
an error, else the statement is executed. Mirroring how Rusqlite does
it, I modified the `OpenFlags` to use bitflags to better configure how
we open our VFS. This modification, will enable us to run tests against
the same database in parallel.

Closes #1433
2025-05-03 19:15:06 +03:00
pedrocarlo
0c22382f3c shared lock on file and throw ReadOnly error in transaction 2025-05-02 16:30:48 -03:00
PThorpe92
1e2be35e3b Add fs feature to rustix dependency 2025-04-29 23:07:28 -04:00
PThorpe92
7b6452034b Bump lru dependency to 0.14.0 2025-04-29 10:44:26 -04:00
PThorpe92
7a3d949bd1 Bump mimalloc dependency to 0.1.46 2025-04-29 10:43:46 -04:00
PThorpe92
f581d1de3a Bump miette dependency to 7.6.0 2025-04-29 10:43:07 -04:00
PThorpe92
ba225ade0d Bump libc dependency to 0.2.172 2025-04-29 10:42:10 -04:00
PThorpe92
582ca68640 Bump rustix dependency to v1.0.5 2025-04-29 10:39:26 -04:00
PThorpe92
2785fd5d4a Bump polling crate dependency to 3.7.4 2025-04-29 10:38:46 -04:00
PThorpe92
be5ae7d0e3 Bump io_uring dependency to 0.7.5 2025-04-29 10:38:01 -04:00
Levy A.
f1ee92bf2d numeric types overhaul 2025-04-23 08:34:58 -03:00