Commit Graph

44 Commits

Author SHA1 Message Date
Piotr Rzysko
977b6b331a Fix memory leak caused by unclosed virtual table cursors
The following code reproduces the leak, with memory usage increasing
over time:

```
#[tokio::main]
async fn main() {
    let db = Builder::new_local(":memory:").build().await.unwrap();
    let conn = db.connect().unwrap();

    conn.execute("SELECT load_extension('./target/debug/liblimbo_series');", ())
        .await
        .unwrap();

    loop {
        conn.execute("SELECT * FROM generate_series(1,10,2);", ())
            .await
            .unwrap();
    }
}
```

After switching to the system allocator, the leak becomes detectable
with Valgrind:

```
32,000 bytes in 1,000 blocks are definitely lost in loss record 24 of 24
   at 0x538580F: malloc (vg_replace_malloc.c:446)
   by 0x62E15FA: alloc::alloc::alloc (alloc.rs:99)
   by 0x62E172C: alloc::alloc::Global::alloc_impl (alloc.rs:192)
   by 0x62E1530: allocate (alloc.rs:254)
   by 0x62E1530: alloc::alloc::exchange_malloc (alloc.rs:349)
   by 0x62E0271: new<limbo_series::GenerateSeriesCursor> (boxed.rs:257)
   by 0x62E0271: open_GenerateSeriesVTab (lib.rs:19)
   by 0x425D8FA: limbo_core::VirtualTable::open (lib.rs:732)
   by 0x4285DDA: limbo_core::vdbe::execute::op_vopen (execute.rs:890)
   by 0x42351E8: limbo_core::vdbe::Program::step (mod.rs:396)
   by 0x425C638: limbo_core::Statement::step (lib.rs:610)
   by 0x40DB238: limbo::Statement::execute::{{closure}} (lib.rs:181)
   by 0x40D9EAF: limbo::Connection::execute::{{closure}} (lib.rs:109)
   by 0x40D54A1: example::main::{{closure}} (example.rs:26)
```

Interestingly, when using mimalloc, neither Valgrind nor mimalloc’s
internal statistics report the leak.
2025-05-05 21:26:23 +02:00
PThorpe92
245e7f94f6 Store packed field on ConstraintInfo to optimize planning for vfilter 2025-04-17 14:01:45 -04:00
PThorpe92
e17fd7edc4 Add comments and address PR review 2025-04-17 14:01:44 -04:00
PThorpe92
de27c2fe4c Properly handle pushing predicates for query optimization from xBestIndex 2025-04-17 14:01:37 -04:00
PThorpe92
853af16946 Implement xBestIndex for virtual table api to improve query planning 2025-04-17 13:53:27 -04:00
pedrocarlo
2181de79de add destroy function to vtab 2025-04-13 17:06:12 -03:00
PThorpe92
ad88a50b6b Add docs to ext readme about vfs feature gate 2025-03-19 10:19:19 -04:00
PThorpe92
57d4aa7216 Reorganize ext library and feature gate vfs to more easily prevent wasm build issues 2025-03-19 10:17:11 -04:00
PThorpe92
216a8e7848 Update getrandom dependency in ext api crate 2025-03-12 21:52:52 -04:00
PThorpe92
2cc72ed9ab Feature flag vfs for fs feature/prevent wasm 2025-03-12 21:52:51 -04:00
PThorpe92
8e2c9367c0 add missing method to add builtin vfs to ext api 2025-03-12 21:52:51 -04:00
PThorpe92
68eca4feed Add demo vfs module to vtab kvstore 2025-03-12 21:52:51 -04:00
PThorpe92
8d3c44cf00 Impl IO trait for VfsMod type 2025-03-12 21:52:50 -04:00
PThorpe92
b2748c61b2 Define API for registration of staticly linked vfs modules 2025-03-12 21:52:50 -04:00
PThorpe92
7c4f5d8df8 Add macros for generating FFI functions to support vfs 2025-03-12 21:52:50 -04:00
PThorpe92
20f92fdacf Define API for vfs modules extensions 2025-03-12 21:52:50 -04:00
PThorpe92
25ed6a2985 Store dynamic ext libs in oncecell to prevent UB 2025-03-12 21:52:50 -04:00
PThorpe92
19d35882f9 Add convenience function for creating json ext value type 2025-03-01 14:34:51 -05:00
PThorpe92
741c4e8037 Add json subtype for extension value text type 2025-03-01 14:27:33 -05:00
PThorpe92
c2fe1f631c Remove confusing comment from ext types 2025-03-01 14:27:33 -05:00
PThorpe92
5b8efd92a4 Update extension ownership cleanups for new vtab module 2025-03-01 14:27:33 -05:00
PThorpe92
e7713e87ec Prevent extensions from accidentally freeing value types, fix comments 2025-03-01 14:27:33 -05:00
PThorpe92
e63436dc47 Fix sqlite_schema and remove explicit vtables 2025-02-17 20:44:45 -05:00
PThorpe92
0547d397b1 Update extension api for vtable interface 2025-02-17 20:44:45 -05:00
PThorpe92
9c8083231c Implement create virtual table and VUpdate opcode 2025-02-17 20:44:44 -05:00
PThorpe92
d775b3ea5a Improve extension API with results, fix paths in proc macros 2025-02-17 20:44:39 -05:00
meteorgan
744df6eedf fix some typo 2025-02-13 23:03:41 +08:00
PThorpe92
cd83ac6146 Remove error from vcursor trait in extensions 2025-02-06 09:15:39 -05:00
PThorpe92
a8ae957162 Add tests for series extension, finish initial vtable impl 2025-02-06 09:15:39 -05:00
PThorpe92
d4c06545e1 Refactor vtable impl and remove Rc Refcell from module 2025-02-06 09:15:39 -05:00
PThorpe92
661c74e338 Apply new planner structure to virtual table impl 2025-02-06 09:15:28 -05:00
Jussi Saurio
f5f77c0bd1 Initial virtual table implementation 2025-02-06 07:51:50 -05:00
Diego Reis
05057a04ac completes crypto extension
It aims to be compatible with https://github.com/nalgeon/sqlean/blob/main/docs/crypto.md
2025-02-06 01:42:47 -03:00
Diego Reis
dd58be3b60 Add basic structure for crypto extension 2025-02-05 23:09:26 -03:00
PThorpe92
793cdf8bad Fix memory issues, make extension types more efficient 2025-01-27 22:30:31 -05:00
PThorpe92
bcd3ae2bd7 Add documentation to core/ext and relevant macros 2025-01-18 16:04:28 -05:00
PThorpe92
9c47379927 Fix deref null ptr in Debug impl on external types 2025-01-18 15:23:33 -05:00
PThorpe92
956320b7d0 Fix scalar API in extensions, add some error handling 2025-01-18 15:19:35 -05:00
PThorpe92
b57308003e Handle freeing memory in finalize, remove unused free fn 2025-01-17 21:54:25 -05:00
PThorpe92
fc82461eff Complete percentile extension, enable col+delimeter args 2025-01-17 21:15:09 -05:00
PThorpe92
44374b9e69 Clean up scalar trait remove unnecessary args method 2025-01-17 14:13:57 -05:00
PThorpe92
0c737d88f7 Support aggregate functions in Extensions 2025-01-17 14:13:57 -05:00
PThorpe92
9b7b794e07 Begin sketching out aggregates api 2025-01-17 14:13:42 -05:00
Pekka Enberg
f83b34287e Move limbo_extension crate to extensions/core 2025-01-16 14:39:12 +02:00