Commit Graph

56 Commits

Author SHA1 Message Date
Pekka Enberg
4142f4f4cb Merge 'Organize extension library and feature gate VFS' from Preston Thorpe
I keep having 3+ PR's in at the same time and always deal with crazy
conflicts because everything in the `ext` library is together in one
file.
This PR moves each category of extension into its own file, and
separates the `vfs` functionality in Core into the `ext/dynamic` module,
so that it can be more easily separated from wasm (or non feature =
"fs") targets to prevent build issues.
The only semantic changes made in this PR is the feature gating of vfs,
the rest is simply organizing and cleaning up imports.
Was unsure if `vfs` should be a feature on the `core` side too, or to
just enable it with the `fs` feature which seemed reasonable, as that
was already the current behavior. But let me know if we want it entirely
behind it's own feature.

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1124
2025-03-19 19:08:13 +02:00
Pekka Enberg
ddb39d2493 Limbo 0.0.17 2025-03-19 17:29:17 +02:00
PThorpe92
ad88a50b6b Add docs to ext readme about vfs feature gate 2025-03-19 10:19:19 -04:00
PThorpe92
57d4aa7216 Reorganize ext library and feature gate vfs to more easily prevent wasm build issues 2025-03-19 10:17:11 -04:00
PThorpe92
216a8e7848 Update getrandom dependency in ext api crate 2025-03-12 21:52:52 -04:00
PThorpe92
2cc72ed9ab Feature flag vfs for fs feature/prevent wasm 2025-03-12 21:52:51 -04:00
PThorpe92
8e2c9367c0 add missing method to add builtin vfs to ext api 2025-03-12 21:52:51 -04:00
PThorpe92
18537ed43e Add documentation/example to extensions/core README.md 2025-03-12 21:52:51 -04:00
PThorpe92
68eca4feed Add demo vfs module to vtab kvstore 2025-03-12 21:52:51 -04:00
PThorpe92
8d3c44cf00 Impl IO trait for VfsMod type 2025-03-12 21:52:50 -04:00
PThorpe92
b2748c61b2 Define API for registration of staticly linked vfs modules 2025-03-12 21:52:50 -04:00
PThorpe92
7c4f5d8df8 Add macros for generating FFI functions to support vfs 2025-03-12 21:52:50 -04:00
PThorpe92
20f92fdacf Define API for vfs modules extensions 2025-03-12 21:52:50 -04:00
PThorpe92
25ed6a2985 Store dynamic ext libs in oncecell to prevent UB 2025-03-12 21:52:50 -04:00
Pekka Enberg
49b75d33ba Limbo 0.0.16 2025-03-05 13:46:17 +02:00
Pekka Enberg
ddb188132c Merge 'Clean up extension types API, introduce json text subtype' from Preston Thorpe
This PR cleans up some comments in the extension API and prevents
extensions themselves from calling 'free' on Value types that are
exposed to the user facing traits, as well as changes the `from_ffi`
method for OwnedValues to take ownership and automatically free the
values to prevent memory leaks.
This PR also finds the name of the `args: &[Value]` argument for scalar
functions in extensions, and uses that in the proc macro, instead of
relying on documentation to communicate that the parameter must be named
`args`.

Closes #1054
2025-03-04 10:24:19 +02:00
PThorpe92
19d35882f9 Add convenience function for creating json ext value type 2025-03-01 14:34:51 -05:00
PThorpe92
741c4e8037 Add json subtype for extension value text type 2025-03-01 14:27:33 -05:00
PThorpe92
c2fe1f631c Remove confusing comment from ext types 2025-03-01 14:27:33 -05:00
PThorpe92
5b8efd92a4 Update extension ownership cleanups for new vtab module 2025-03-01 14:27:33 -05:00
PThorpe92
e7713e87ec Prevent extensions from accidentally freeing value types, fix comments 2025-03-01 14:27:33 -05:00
pedrocarlo
afb72b4cd4 minor improvements to extension docs 2025-02-26 13:52:39 -03:00
Pekka Enberg
7f2525ac27 Merge 'Implement create virtual table using vtab modules, more work on virtual tables' from Preston Thorpe
This PR started out as one to improve the API of extensions but I ended
up building on top of this quite a bit and it just kept going. Sorry
this one is so large but there wasn't really a good stopping point, as
it kept leaving stuff in broken states.
**VCreate**: Support for `CREATE VIRTUAL TABLE t USING vtab_module`
**VUpdate**: Support for `INSERT` and `DELETE` methods on virtual
tables.
Sqlite uses `xUpdate` function with the `VUpdate` opcode to handle all
insert/update/delete functionality in virtual tables..
have to just document that:
```
if args[0] == NULL:  INSERT args[1] the values in args[2..]

if args[1] == NULL: DELETE args[0]

if args[0] != NULL && len(args) > 2: Update values=args[2..]  rowid=args[0]
```
I know I asked @jussisaurio on discord about this already, but it just
sucked so bad that I added some internal translation so we could expose
a [nice API](https://github.com/tursodatabase/limbo/pull/996/files#diff-
3e8f8a660b11786745b48b528222d11671e9f19fa00a032a4eefb5412e8200d1R54) and
handle the logic ourselves while keeping with sqlite's opcodes.
I'll change it back if I have to, I just thought it was genuinely awful
to have to rely on comments to explain all that to extension authors.
The included extension is not meant to be a legitimately useful one, it
is there for testing purposes. I did something similar in #960 using a
test extension, so I figure when they are both merged, I will go back
and combine them into one since you can do many kinds at once, and that
way it will reduce the amount of crates and therefore compile time.
1. Remaining opcodes.
2. `UPDATE` (when we support the syntax)
3. `xConnect` - expose API for a DB connection to a vtab so it can
perform arbitrary queries.

Closes #996
2025-02-25 15:31:12 +02:00
Pekka Enberg
11a93900e3 extensions/core: Fix mimalloc version in README.md 2025-02-18 20:07:45 +02:00
Pekka Enberg
9b84e7e919 extensions/core: Add description to Cargo.toml 2025-02-18 19:34:32 +02:00
PThorpe92
e63436dc47 Fix sqlite_schema and remove explicit vtables 2025-02-17 20:44:45 -05:00
PThorpe92
38e54ca85e Update schema dot command to show virtual tables 2025-02-17 20:44:45 -05:00
PThorpe92
0547d397b1 Update extension api for vtable interface 2025-02-17 20:44:45 -05:00
PThorpe92
9c8083231c Implement create virtual table and VUpdate opcode 2025-02-17 20:44:44 -05:00
PThorpe92
d775b3ea5a Improve extension API with results, fix paths in proc macros 2025-02-17 20:44:39 -05:00
Pekka Enberg
43574528c9 Merge 'Switch to workspace dependencies' from Pekka Enberg
...makes it easier to specify a version, which is needed for `cargo
publish`.

Closes #991
2025-02-14 12:21:24 +02:00
meteorgan
744df6eedf fix some typo 2025-02-13 23:03:41 +08:00
Pekka Enberg
ac54c35f92 Switch to workspace dependencies
...makes it easier to specify a version, which is needed for `cargo publish`.
2025-02-12 17:28:04 +02:00
meteorgan
a99d9a8988 chore: remove unused dependencies 2025-02-09 21:05:06 +08:00
PThorpe92
cd83ac6146 Remove error from vcursor trait in extensions 2025-02-06 09:15:39 -05:00
PThorpe92
a8ae957162 Add tests for series extension, finish initial vtable impl 2025-02-06 09:15:39 -05:00
PThorpe92
ad30ccdc0e Add docs in extension README for vtable modules 2025-02-06 09:15:39 -05:00
PThorpe92
d4c06545e1 Refactor vtable impl and remove Rc Refcell from module 2025-02-06 09:15:39 -05:00
PThorpe92
661c74e338 Apply new planner structure to virtual table impl 2025-02-06 09:15:28 -05:00
Jussi Saurio
f5f77c0bd1 Initial virtual table implementation 2025-02-06 07:51:50 -05:00
Diego Reis
05057a04ac completes crypto extension
It aims to be compatible with https://github.com/nalgeon/sqlean/blob/main/docs/crypto.md
2025-02-06 01:42:47 -03:00
Diego Reis
dd58be3b60 Add basic structure for crypto extension 2025-02-05 23:09:26 -03:00
PThorpe92
793cdf8bad Fix memory issues, make extension types more efficient 2025-01-27 22:30:31 -05:00
PThorpe92
f13d035965 Enable wasm to static link extensions 2025-01-21 09:36:49 -05:00
PThorpe92
bcd3ae2bd7 Add documentation to core/ext and relevant macros 2025-01-18 16:04:28 -05:00
PThorpe92
9c47379927 Fix deref null ptr in Debug impl on external types 2025-01-18 15:23:33 -05:00
PThorpe92
956320b7d0 Fix scalar API in extensions, add some error handling 2025-01-18 15:19:35 -05:00
Jussi Saurio
a48c9033e2 Merge pull request #724 from PThorpe92/extdocs
Add documentation for extensions/core
2025-01-18 11:08:35 +02:00
PThorpe92
b57308003e Handle freeing memory in finalize, remove unused free fn 2025-01-17 21:54:25 -05:00
PThorpe92
fc82461eff Complete percentile extension, enable col+delimeter args 2025-01-17 21:15:09 -05:00