Commit Graph

260 Commits

Author SHA1 Message Date
pedrocarlo
2d6fad5ea3 nit: adjust order of struct completions 2025-08-19 10:48:21 -03:00
pedrocarlo
fadf78fe67 use a dedicated Error enum for Completion Error 2025-08-19 10:48:21 -03:00
pedrocarlo
7bc0545442 default impl for get_memory_io 2025-08-19 10:48:21 -03:00
pedrocarlo
d5a59c6bee default impl for generate_random_number 2025-08-19 10:48:21 -03:00
pedrocarlo
f72bcbc5da default impl for wait_for_completion + check for errors in completion there 2025-08-19 10:48:21 -03:00
pedrocarlo
002390b5a5 store error inside Completion 2025-08-19 10:48:21 -03:00
pedrocarlo
d0c13f0104 remove IOError from Parser + store only ErrorKind in LimboError 2025-08-19 10:48:21 -03:00
PThorpe92
d3d01cefc8 Add to_system_time for our io::clock::Instant type 2025-08-18 19:27:37 -04:00
PThorpe92
cc2fed3297 Remove copy_to API from file IO trait 2025-08-14 21:31:13 -04:00
PThorpe92
55f09a01c4 Update copy_to method in file trait to separate source and destination IO 2025-08-14 21:31:13 -04:00
Preston Thorpe
5cea0f572e Merge 'Revive async io extension PR' from Preston Thorpe
bringing #1127 back to life, except better because this doesn't add
Tokio as a dependency for extension lib just for tests.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2418
2025-08-14 16:10:09 -04:00
Preston Thorpe
e9d23e7fea apply copilot suggestion
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-08-14 16:08:55 -04:00
Jussi Saurio
a2a6feb193 Merge 'Use BufferPool owned by Database instead of a static global' from Jussi Saurio
## Problem
There are several problems with our current statically allocated
`BufferPool`.
1. You cannot open two databases in the same process with different page
sizes, because the `BufferPool`'s `Arena`s will be locked forever into
the page size of the first database. This is the case regardless of
whether the two `Database`s are open at the same time, or if the first
is closed before the second is opened.
2. It is impossible to even write Rust tests for different page sizes
because of this, assuming the test uses a single process.
## Solution
Make `Database` own `BufferPool` instead of it being statically
allocated, so this problem goes away.
Note that I didn't touch the still statically-allocated
`TEMP_BUFFER_CACHE`, because it should continue to work regardless of
this change. It should only be a problem if the user has two or more
databases with different page sizes open simultaneously, because
`TEMP_BUFFER_CACHE` will only support one pool of a given page size at a
time, so the rest of the allocations will go through the global
allocator instead.
## Notes
I extracted this change out from #2569, because I didn't want it to be
smuggled in without being reviewed as an individual piece.

Reviewed-by: Avinash Sajjanshetty (@avinassh)

Closes #2596
2025-08-14 12:40:32 +03:00
Jussi Saurio
78f1ed979e Merge 'io_uring: Gracefully handle submission queue overflow' from Preston Thorpe
Current handling is not ideal, this adds proper squeue overflow handling
by ensuring everything is still submitted in-order

Closes #2586
2025-08-14 10:55:17 +03:00
Jussi Saurio
359cba0474 Use BufferPool owned by Database instead of a static global
Problem

There are several problems with our current statically allocated
`BufferPool`.

1. You cannot open two databases in the same process with different
page sizes, because the `BufferPool`'s `Arena`s will be locked forever
into the page size of the first database. This is the case regardless
of whether the two `Database`s are open at the same time, or if the first
is closed before the second is opened.

2. It is impossible to even write Rust tests for different page sizes because
of this, assuming the test uses a single process.

Solution

Make `Database` own `BufferPool` instead of it being statically allocated, so this
problem goes away.

Note that I didn't touch the still statically-allocated `TEMP_BUFFER_CACHE`, because
it should continue to work regardless of this change. It should only be a problem if
the user has two or more databases with different page sizes open simultaneously, because
`TEMP_BUFFER_CACHE` will only support one pool of a given page size at a time, so the rest
of the allocations will go through the global allocator instead.

Notes

I extracted this change out from #2569, because I didn't want it to be smuggled in without
being reviewed as an individual piece.
2025-08-14 10:29:52 +03:00
PThorpe92
3840708970 Add safety comment 2025-08-13 21:42:19 -04:00
PThorpe92
0e3750b85b Integrate changes to vfs IO impl for callbacks 2025-08-13 21:42:18 -04:00
PThorpe92
ec4bf19fc7 Gracefully handle submission queue overflow in io_uring backend 2025-08-13 12:07:41 -04:00
Nikita Sivukhin
c3c7463193 add debug logs to the memory io 2025-08-13 15:23:18 +04:00
pedrocarlo
2e68296107 create IOCompletions 2025-08-13 10:24:55 +03:00
PThorpe92
213d589dd1 Apply review suggestions, remove FreeEntry 2025-08-08 11:07:29 -04:00
PThorpe92
faf248df03 Add more docs and comments for TempBufferCache 2025-08-08 10:55:28 -04:00
PThorpe92
34d90d5acb Remove Clone impl for Buffer and PageContent to make any copying of page data explicit 2025-08-08 10:55:28 -04:00
PThorpe92
d7e4ba21f8 Add explanation for using 3mb limit 2025-08-08 10:55:28 -04:00
PThorpe92
66964fd8d2 Add documentation and comments to new buffer pool, add get_frame api 2025-08-08 10:55:28 -04:00
PThorpe92
d94e252ef9 Actually pre-register two arenas with the ring 2025-08-08 10:55:28 -04:00
PThorpe92
39d230a899 Add bitmap for tracking pages in arena 2025-08-08 10:55:27 -04:00
PThorpe92
0ffba81216 Make register buffer io trait return the buf index 2025-08-08 10:55:27 -04:00
PThorpe92
fd09fe1237 Adjust io_uring to register two arenas, one for frames and the other for db pages 2025-08-08 10:55:27 -04:00
PThorpe92
cc75bc448e Move TLC buffer cache to io/mod 2025-08-08 10:55:27 -04:00
PThorpe92
9d1ca1c8ca Add ReadFixed/WriteFixed opcodes for buffers from registered arena 2025-08-08 10:55:27 -04:00
PThorpe92
a02f527c06 Add fast path for pwritev on other IO backends 2025-08-08 10:55:25 -04:00
PThorpe92
39bccc2357 Update Buffer type in io module to adjust to new pool 2025-08-08 10:55:25 -04:00
PThorpe92
bcadcb2014 Remove RefCell from copy_to method in io trait 2025-08-07 17:07:53 -04:00
PThorpe92
039fe22405 Add copy_to to io::File trait to support copying DB files 2025-08-07 16:27:02 -04:00
PThorpe92
f6fb786cc9 Fix borrow method on WindowsIO 2025-08-05 22:26:19 -04:00
PThorpe92
53a0524050 Fix clippy warning 2025-08-05 16:24:50 -04:00
PThorpe92
f6a68cffc2 Remove RefCell from IO and Page apis 2025-08-05 16:24:49 -04:00
PThorpe92
914c10e095 Remove Clone impl for Buffer and PageContent 2025-08-05 14:26:53 -04:00
Pekka Enberg
aa20c2f1ba Merge 'Relax I/O configuration attribute to cover all Unixes' from Pedro Muniz
hopefully fixes #2268.

Closes #2435
2025-08-05 12:44:34 +03:00
PThorpe92
2a3fa0955f Attempt to coalesce contiguous iovecs during pwritev operation for unix IO 2025-08-04 16:18:19 -04:00
PThorpe92
b76ef20f4c Attempt to coalesce contiguous iovecs during pwritev operation for io_uring 2025-08-04 16:18:05 -04:00
pedrocarlo
ebe6aa0d28 adjust cfg for unix and linux IO 2025-08-04 15:49:52 -03:00
Jussi Saurio
1813171b91 Merge 'Use pwrite for single buffer pwritev call in unix IO' from Preston Thorpe
Closes #2416
2025-08-04 16:52:14 +03:00
Nikita Sivukhin
129895f0b2 preserve files in IO memory backend 2025-08-04 15:22:04 +04:00
PThorpe92
79629daff4 Make completions idempotent 2025-08-02 21:48:39 -04:00
PThorpe92
b5117ac5c7 Use pwrite for single buffer in unix IO 2025-08-02 18:34:16 -04:00
PThorpe92
b8ed4358f1 register buffers sparse on ring initiate to support fixed operations 2025-08-01 14:56:43 -04:00
PThorpe92
9289dd7e9a Implement register_fixed_buffer for io_uring IO backend 2025-08-01 14:55:35 -04:00
PThorpe92
3048e4fa97 Add optional register_fixed_buffer method to IO trait 2025-08-01 14:54:26 -04:00