Commit Graph

1039 Commits

Author SHA1 Message Date
Pekka Enberg
d355ce785c core/storage: Remove debug printout 2025-01-14 17:54:17 +02:00
Jussi Saurio
fcfee24c50 Remove mark_last_insn_constant() from places where it is not safe to do so 2025-01-14 16:06:49 +02:00
PThorpe92
9c208dc866 Add tests for first extension 2025-01-14 07:27:35 -05:00
PThorpe92
e4ce6402eb Remove previous uuid implementation 2025-01-14 07:20:50 -05:00
PThorpe92
3099e5c9ba Improve api, standardize conversions between types, finish extension 2025-01-14 07:20:50 -05:00
PThorpe92
6e05258d36 Add safety comments and clean up extension types 2025-01-14 07:20:50 -05:00
PThorpe92
98eff6cf7a Enable passing arguments to external functions 2025-01-14 07:20:50 -05:00
PThorpe92
c565fba195 Adjust types in extension API 2025-01-14 07:20:49 -05:00
PThorpe92
3412a3d4c2 Rough design for extension api/draft extension 2025-01-14 07:20:48 -05:00
PThorpe92
0a10d893d9 Sketch out runtime extension loading 2025-01-14 07:18:07 -05:00
Pekka Enberg
537aea590f Merge 'Clean up license management ' from Kim Seon Woo
## Purpose of this PR
- Set up on how to manage licenses in limbo project
## Changes
- Move all licenses under `licenses` directory in the root path
- Add `NOTICE.md`
- Update `CONTRIBUTING.md` to explain how to add licenses to project
## Reference
https://github.com/tursodatabase/limbo/issues/615

Closes #677
2025-01-14 14:15:53 +02:00
김선우
8ed2d14ca4 Reposition licenses in core package 2025-01-14 19:42:09 +09:00
Jorge López
a16282ea62 core: remove nix as a dependency 2025-01-14 11:06:13 +01:00
Jorge López
55e06b0c72 core/io: make file locks non-blocking so they fail right away 2025-01-14 11:01:11 +01:00
Jussi Saurio
d3c2868fcb Merge 'refactor: json functions vdbe code should be consistent with scalar functions' from Jorge Hermo
While working to include new json functions (related to #127) I noticed
that the program step code for json functions https://github.com/tursoda
tabase/limbo/blob/0dceb02ec04241b3772332853bcbfb9eb559adb9/core/vdbe/mod
.rs#L1346 was a bit different from scalar functions's code, where the
match to the inner function is nested https://github.com/tursodatabase/l
imbo/blob/0dceb02ec04241b3772332853bcbfb9eb559adb9/core/vdbe/mod.rs#L142
4
I added the same nesting to the json functions so it is more consistent

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #663
2025-01-14 10:30:33 +02:00
Pekka Enberg
5c9505e8f7 Revert "core/io/io_uring: replace nix and libc calls with their rustix counterparts."
This reverts commit b146f5d4cb because it
causes tests to hang.
2025-01-14 10:25:23 +02:00
Pekka Enberg
d223c72d03 Revert "core: Previous commits didn't actually remove nix as dependency, so do that here"
This reverts commit cca3846f95, we need to
bring it back unfortunately.
2025-01-14 10:25:11 +02:00
Pekka Enberg
945a91dee5 Merge 'core: Consolidate libc implementations' from Jorge López Tello
This shaves off `nix` as a dependency in core, which was only being used
in the io_uring backend for the `fcntl` advisory record locking. Since a
previous PR made `rustix` a dep not only for Mac but for any Unix, and
`rustix` also has `fcntl`, `nix` becomes redundant.
Furthermore, it reduces `libc` usage across core. The only remaining
uses are:
```rust
io_uring::opcode::Readv::new(fd, iovec as *const iovec as *const libc::iovec, 1)
io_uring::opcode::Writev::new(fd, iovec as *const iovec as *const libc::iovec, 1)
```
These two are a little ugly, but sadly the `io_uring` crate requires
both opcodes to take a `libc::iovec` while it doesn't export the type.
See tokio-rs/io-uring#259 for a request to use `std::io::IoSlice` or to
export the type directly.
Apart from those two, there are no other usages of libc, so if those are
resolved, we could also drop the dependency on libc.

Closes #668
2025-01-14 10:07:20 +02:00
Pekka Enberg
f1b6003d2a Merge 'Fix the rest of the flaky datetime tests' from Preston Thorpe
seeing #660 made me realize I shouldn't have used `now` in any
circumstances in these tests, tolerance or not ;)
they should all be deterministic now

Closes #671
2025-01-14 09:18:17 +02:00
PThorpe92
abb3fda19f Fix all the flaky datetime tests 2025-01-13 19:51:34 -05:00
Levy A.
2f2c96fa2c chore: cargo fmt 2025-01-13 21:31:33 -03:00
Levy A.
eff5de50c5 refactor: make translate_* functions accept ProgramBuilder
simplifies function signatures and allows attaching more context to
ProgramStatus on `translate::translate`, useful for query parameters.
2025-01-13 20:41:56 -03:00
Jorge Hermo
df3123e128 Merge with main 2025-01-13 22:20:17 +01:00
Jorge López
cca3846f95 core: Previous commits didn't actually remove nix as dependency, so do that here 2025-01-13 21:15:36 +01:00
Jorge López
2f90a06533 core/io/unix: replace O_NONBLOCK flag from libc with equivalent from rustix 2025-01-13 21:03:05 +01:00
Jorge López
5e9cb58f04 core/io/io_uring: remove unnecessary path prefix for log macros, and replace one unwrap with ? 2025-01-13 20:21:37 +01:00
Jorge López
b146f5d4cb core/io/io_uring: replace nix and libc calls with their rustix counterparts.
core: remove dependency on nix. We keep depending on libc, though, because crate io_uring requires libc's iovec.
2025-01-13 20:21:37 +01:00
Jorge López
b1e8f2da73 core/io/unix: minor formatting 2025-01-13 20:21:37 +01:00
Jorge López
7b5e5efd14 core/io/unix: replace libc calls and types with their rustix counterparts 2025-01-13 20:21:37 +01:00
Jorge López
7808665c92 core: make MAX_IOVECS u32 instead of usize, to match the type expected by io_uring 2025-01-13 20:21:37 +01:00
Jorge López
d4de451d45 core: enable rustix/io_uring with io_uring feature 2025-01-13 20:21:15 +01:00
Pekka Enberg
96148af331 Merge 'Fix 8-bit serial type to encoding' from Preston Thorpe
This PR fixes #666
Limbo was treating all single byte integers properly in `core/types.rs`,
but when converted in `sqlite3_ondisk`, it was losing the sign for 8 bit
integers, treating them all as unsigned. Sqlite3 specifies in their file
format that the `1` record serial type is "Value is an 8-bit twos-
complement integer."  https://www.sqlite.org/fileformat.html
We now properly match sqlite3's output
![image](https://github.com/user-attachments/assets/567960ca-
bfc6-4210-9a0e-ae4203a63add)
![image](https://github.com/user-
attachments/assets/ed4e709b-63ea-4de9-bbdc-a3f8b4dfd0f9)

Closes #667
2025-01-13 20:59:54 +02:00
Pekka Enberg
42293115de Merge 'cli: Introduce a selectable IO backend with --io={syscall,io-uring} argument' from Jorge López Tello
Give the user the option of choosing IO backend. There is only a
"syscall" backend, unless built for Linux with `#[cfg(feature =
"io_uring")]` which introduces `--io=io-uring`.
Right now, the choice has only been implemented for the CLI. Bindings
and such default to PlatformIO, except when an "in-memory" database has
been chosen.
Can be tested by running CLI with RUST_LOG=debug

Reviewed-by: Preston Thorpe <preston@unlockedlabs.org>

Closes #654
2025-01-13 20:59:20 +02:00
PThorpe92
0b6061de2f Adjust sqlite serial type to account for single byte signed integer 2025-01-13 13:34:46 -05:00
Pekka Enberg
44550ab749 cargo fmt 2025-01-13 18:58:40 +02:00
Pekka Enberg
c7ea2393b3 Merge 'Store cursor types in program state and remove trait Cursor' from Jussi Saurio
I was planning on starting work on index insertion, but realized we need
to know whether our cursor refers to a table or an index etc., so it
resulted in this refactoring work.
- `cursor_ref` now contains what _type_ of cursor it is (table, index,
pseudo, sorter)
- `program.cursors` is now `program.btree_table_cursors`,
`program.btree_index_cursors` etc and they are unboxed because dynamic
dispatch is no longer necessary
- Cursor trait removed -- 95% of the shit was btree specific anyway, so
I just moved them to `BTreeCursor`. In certain instructions in the VDBE
we expect a btree cursor and in others we expect a pseudo/sorter etc,
lets make that explicit.
- I also removed `BTreeCursor::get_new_rowid()` specific tests that
required macros to generate a mock implementation of the `Cursor` trait
-- main reason is I couldn't figure out how to reimplement this without
the trait, and the second reason is that I don't think we really need
those tests, AND the proc macro is constantly failing in my editor as
well and screwing up `rust-analyzer`

Closes #655
2025-01-13 18:40:46 +02:00
Pekka Enberg
8769bf82a5 Merge 'fix: flaky test in datetime because compare pre-init now.' from Sonny
Removed flaky test that happens sometimes when the CI runner is a bit
slow e.g. in https://github.com/tursodatabase/limbo/actions/runs/1270626
5753/job/35418879041?pr=643
'now' is covered in other test so it should be fine. it is a problem in
time test, for date not so much.
for example it is covered in:
https://github.com/tursodatabase/limbo/blob/bcc85c37a0febdf5c3f29127029f
e10c9906867d/core/vdbe/datetime.rs#L1434C5-L1446C6

Closes #652
2025-01-13 18:39:54 +02:00
Pekka Enberg
1e94dbffcc Merge branch 'main' into json-error-position 2025-01-13 18:21:37 +02:00
Jorge Hermo
016d9d17ab refactor: json functions vdbe 2025-01-13 00:05:14 +01:00
Jorge López Tello
7365bee984 Merge branch 'tursodatabase:main' into selectable-io-backend 2025-01-12 15:11:54 +01:00
Jussi Saurio
92888c54e9 remove get_new_rowid() specific tests that require mocking 2025-01-11 17:41:10 +02:00
Jussi Saurio
bf48c0ae72 Remove trait Cursor 2025-01-11 17:19:25 +02:00
Jorge López
5a45df84db core: add debug line when an IO backend is created. A user can check that the correct one has been selected. 2025-01-11 16:08:41 +01:00
Jussi Saurio
9f44d2a7ac Remove unused Table::Index variant and unused Table methods 2025-01-11 17:07:30 +02:00
Jussi Saurio
9909539b9d Store cursor type (table,index,pseudo,sorter) when allocating cursor 2025-01-11 17:04:16 +02:00
Jorge López
2596e0800e core: expose UnixIO apart from PlatformIO on any Unix, and expose UringIO on Linux with feature io_uring.
cli: add a new argument to select I/O backend (more than one option only for Linux with io_uring feature).

cli: make both Limbo::new() and Limbo::open_db() use get_io(), unifying parsing of database path and eliminating duplicated code.
2025-01-11 15:21:26 +01:00
Jorge López
e5bd39a8b2 Tiny formatting change, because RustRover keeps redoing it. Having to revert on every commit is not fun. 2025-01-11 15:03:26 +01:00
sonhmai
46eba6cc77 fix: flaky test in datetime because compare pre-init now.
'now' is covered in other test so it should be fine.
2025-01-11 16:15:58 +07:00
Jussi Saurio
e782f30ffe Add expect_arguments_min!() and use expect_arguments_* macros more universally 2025-01-11 09:55:07 +02:00
Peter Sooley
b5fed15997 implement json_error_position 2025-01-10 11:12:30 -08:00