Commit Graph

137 Commits

Author SHA1 Message Date
Pekka Enberg
e21e2b5449 Merge 'core: Fix syscall VFS on Linux' from Pekka Enberg
Fix the syscall VFS on Linux not to use `PlatformIO`, which is just an
alias for `io_uring`.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #1285
2025-04-09 19:38:45 +03:00
Pekka Enberg
2d009083ba core: Fix syscall VFS on Linux
Fix the syscall VFS on Linux not to use `PlatformIO`, which is just an
alias for `io_uring`.
2025-04-09 19:27:58 +03:00
Avinash Sajjanshetty
2873c36b31 Implement trait Clock for implemented IOs
Replace `chrono::Local::now()` to return `Instant` but containing
same info
2025-04-06 23:34:13 +05:30
Avinash Sajjanshetty
8d6d50d9d5 Update IO trait
- Remove existing `get_current_time() -> String`
- Add super trait bound `Clock`
2025-04-06 23:06:57 +05:30
Avinash Sajjanshetty
02ee2cf90e Add Clock trait 2025-04-06 23:02:41 +05:30
PThorpe92
e9420e7d2b Fix platform specific ffi c ptr types 2025-03-24 22:48:07 -04:00
Daniel Boll
fc50609491 Fix io_uring WAL write corruption by ensuring buffer lifetime
Ensure the Arc<RefCell<Buffer>> in UringFile::pwrite remains alive until
the io_uring write completes by referencing it in the completion callback.
This prevents WAL file corruption where the correct buffer data was
overwritten with stale memory (e.g., 00 18 27 xx instead of 37 7f 06 82).

Validation:
- Tested with limbo -v io_uring and WAL operations.
- Verified with xxd and wal-browser.

Signed-off-by: Daniel Boll <danielboll.academico@gmail.com>
2025-03-19 23:55:43 -03:00
PThorpe92
57d4aa7216 Reorganize ext library and feature gate vfs to more easily prevent wasm build issues 2025-03-19 10:17:11 -04:00
PThorpe92
8e4705d6c6 Fix read completion error in vfs 2025-03-14 18:08:49 -04:00
PThorpe92
44f6054657 Impl copy + clone for io openflags 2025-03-12 21:52:51 -04:00
PThorpe92
8d3c44cf00 Impl IO trait for VfsMod type 2025-03-12 21:52:50 -04:00
Pere Diaz Bou
b63f541150 impl Send Sync for GenericFile 2025-03-05 14:07:48 +01:00
Pere Diaz Bou
d1e7f0826b impl Send Sync for WindowsFile 2025-03-05 14:07:48 +01:00
Pere Diaz Bou
3d3cdbeb0c fix generic and windows sync compilation 2025-03-05 14:07:48 +01:00
Pere Diaz Bou
ab2fc3fb13 fix clippy arc warnings for now 2025-03-05 14:07:48 +01:00
Pere Diaz Bou
d1c7d758c4 fix uring Sync,Sync unsafe impl 2025-03-05 14:07:48 +01:00
Pere Diaz Bou
e4a8ee5402 move load extensions to Connection
Extensions are loaded per connection and not per database as per SQLite
behaviour. This also helps with removing locks.
2025-03-05 14:07:48 +01:00
Pere Diaz Bou
8daf7666d1 Make database Sync + Send 2025-03-05 14:07:48 +01:00
Pekka Enberg
01671d1d1e Merge 'Performance improvements for Unix I/O backend' from Preston Thorpe
This PR reworks the unix I/O backend, removing runtime reference
counting/borrow checking and optimizing away the hashmap in favor of a
static array, with an unlikely fallback vec.
The only reason the fallback vec is there is because unlike the
`io_uring` module, we cannot simply index into the array with the fd  as
the OS could theoretically give us a fd up to I believe 1024 so keeping
an array of that size for a few elements is unnecessary.

Closes #940
2025-02-12 09:29:41 +02:00
PThorpe92
e3ab80ae28 Replace Option with MaybeUninit in unix IO 2025-02-11 09:09:49 -05:00
PThorpe92
9098237a12 Add as_read method to completion enum 2025-02-11 09:09:47 -05:00
PThorpe92
975cf95c37 Reformat unix io with wrappers for unsafecell 2025-02-11 09:06:49 -05:00
PThorpe92
159e2fbd71 Remove rc refcell in unix io module 2025-02-11 09:01:57 -05:00
Doug Anderson444
0aa5de6d9f rm log, switch all to tracing 2025-02-11 09:03:36 -04:00
Pekka Enberg
0638550be7 Merge 'Remove unnecessary reference counting from completion I/O callbacks' from Preston Thorpe
I am on a bit of a mission to revisit a lot of the ref counting, this
was an easy first win.
It seems to be a linear path of function calls or hashmaps which can own
the completions directly, no cloning needed.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #912
2025-02-10 12:11:30 +02:00
Jorge López
83b158fb3a core: silence some unused warnings when building without default features 2025-02-09 01:13:12 +01:00
PThorpe92
75898027a0 Remove unnecessary reference counting from completion io callbacks 2025-02-08 08:20:19 -05:00
PThorpe92
575a524d04 Replace hashmap for io_uring pending ops with static array 2025-02-07 09:05:08 -05:00
PThorpe92
b5a5b34329 Use Cell for size in memoryio in place of unsafecell 2025-02-01 23:04:42 -05:00
PThorpe92
5c00b3dc6e Improve in-memory IO performance, remove runtime borrow checking 2025-02-01 21:54:39 -05:00
Jorge López
55e06b0c72 core/io: make file locks non-blocking so they fail right away 2025-01-14 11:01:11 +01:00
Pekka Enberg
5c9505e8f7 Revert "core/io/io_uring: replace nix and libc calls with their rustix counterparts."
This reverts commit b146f5d4cb because it
causes tests to hang.
2025-01-14 10:25:23 +02:00
Jorge López
2f90a06533 core/io/unix: replace O_NONBLOCK flag from libc with equivalent from rustix 2025-01-13 21:03:05 +01:00
Jorge López
5e9cb58f04 core/io/io_uring: remove unnecessary path prefix for log macros, and replace one unwrap with ? 2025-01-13 20:21:37 +01:00
Jorge López
b146f5d4cb core/io/io_uring: replace nix and libc calls with their rustix counterparts.
core: remove dependency on nix. We keep depending on libc, though, because crate io_uring requires libc's iovec.
2025-01-13 20:21:37 +01:00
Jorge López
b1e8f2da73 core/io/unix: minor formatting 2025-01-13 20:21:37 +01:00
Jorge López
7b5e5efd14 core/io/unix: replace libc calls and types with their rustix counterparts 2025-01-13 20:21:37 +01:00
Jorge López
7808665c92 core: make MAX_IOVECS u32 instead of usize, to match the type expected by io_uring 2025-01-13 20:21:37 +01:00
Jorge López
5a45df84db core: add debug line when an IO backend is created. A user can check that the correct one has been selected. 2025-01-11 16:08:41 +01:00
Jorge López
2596e0800e core: expose UnixIO apart from PlatformIO on any Unix, and expose UringIO on Linux with feature io_uring.
cli: add a new argument to select I/O backend (more than one option only for Linux with io_uring feature).

cli: make both Limbo::new() and Limbo::open_db() use get_io(), unifying parsing of database path and eliminating duplicated code.
2025-01-11 15:21:26 +01:00
Jorge López
511c0b495d Rename LinuxIOError to UringIOError to match the IO backend renames 2025-01-07 15:19:31 +01:00
Jorge López
e5a12bdf01 Rename linux backend to io_uring and darwin to unix. Add new feature flag to IO backend selection 2025-01-07 15:19:29 +01:00
psvri
e7d4fa0a53 Minor clippy fixes 2025-01-01 16:11:52 +05:30
PThorpe92
4e77840ee5 Setup io_uring with sqpoll enabled 2024-12-29 15:34:17 -05:00
adamnemecek
97647ff056 Clean up code to use Self
Closes #556
2024-12-29 10:07:38 +02:00
Dezhi Wu
bea49549ed feat(core/io): Add support for file creation in open_file function
`cargo test` is always failing on FreeBSD, the following is one of the
errors:

```
---- tests::test_simple_overflow_page stdout ----
thread 'tests::test_simple_overflow_page' panicked at test/src/lib.rs:32:84:
called `Result::unwrap()` on an `Err` value: IOError(Os { code: 2, kind: NotFound, message: "No such file or directory" })
```

After some digging, I found that the `open_file` function in
`core/io/generic.rs` does not respect the `OpenFlags::Create` flag. This
commit adds support for file creation in the `open_file` function.

`cargo test` now passes on FreeBSD.
2024-12-22 11:04:15 +08:00
Ziyak Jehangir
99d1b0e5a3 cleanup: replace &(*x) with x.as_ref() for smart pointer derefs 2024-12-19 19:36:04 +05:30
PThorpe92
7fc9835899 Optimize replace byte vec with boxed array in memory page 2024-12-14 23:39:51 -05:00
PThorpe92
ce2851ac8d Fix properly opening new connection in shell 2024-12-14 21:20:10 -05:00
PThorpe92
89819eebf6 Expose memoryIO from limbo_core 2024-12-14 20:52:45 -05:00