Jussi Saurio
359cba0474
Use BufferPool owned by Database instead of a static global
...
Problem
There are several problems with our current statically allocated
`BufferPool`.
1. You cannot open two databases in the same process with different
page sizes, because the `BufferPool`'s `Arena`s will be locked forever
into the page size of the first database. This is the case regardless
of whether the two `Database`s are open at the same time, or if the first
is closed before the second is opened.
2. It is impossible to even write Rust tests for different page sizes because
of this, assuming the test uses a single process.
Solution
Make `Database` own `BufferPool` instead of it being statically allocated, so this
problem goes away.
Note that I didn't touch the still statically-allocated `TEMP_BUFFER_CACHE`, because
it should continue to work regardless of this change. It should only be a problem if
the user has two or more databases with different page sizes open simultaneously, because
`TEMP_BUFFER_CACHE` will only support one pool of a given page size at a time, so the rest
of the allocations will go through the global allocator instead.
Notes
I extracted this change out from #2569 , because I didn't want it to be smuggled in without
being reviewed as an individual piece.
2025-08-14 10:29:52 +03:00
pedrocarlo
2e68296107
create IOCompletions
2025-08-13 10:24:55 +03:00
PThorpe92
213d589dd1
Apply review suggestions, remove FreeEntry
2025-08-08 11:07:29 -04:00
PThorpe92
faf248df03
Add more docs and comments for TempBufferCache
2025-08-08 10:55:28 -04:00
PThorpe92
34d90d5acb
Remove Clone impl for Buffer and PageContent to make any copying of page data explicit
2025-08-08 10:55:28 -04:00
PThorpe92
d7e4ba21f8
Add explanation for using 3mb limit
2025-08-08 10:55:28 -04:00
PThorpe92
0ffba81216
Make register buffer io trait return the buf index
2025-08-08 10:55:27 -04:00
PThorpe92
cc75bc448e
Move TLC buffer cache to io/mod
2025-08-08 10:55:27 -04:00
PThorpe92
a02f527c06
Add fast path for pwritev on other IO backends
2025-08-08 10:55:25 -04:00
PThorpe92
39bccc2357
Update Buffer type in io module to adjust to new pool
2025-08-08 10:55:25 -04:00
PThorpe92
bcadcb2014
Remove RefCell from copy_to method in io trait
2025-08-07 17:07:53 -04:00
PThorpe92
039fe22405
Add copy_to to io::File trait to support copying DB files
2025-08-07 16:27:02 -04:00
PThorpe92
53a0524050
Fix clippy warning
2025-08-05 16:24:50 -04:00
PThorpe92
f6a68cffc2
Remove RefCell from IO and Page apis
2025-08-05 16:24:49 -04:00
PThorpe92
914c10e095
Remove Clone impl for Buffer and PageContent
2025-08-05 14:26:53 -04:00
pedrocarlo
ebe6aa0d28
adjust cfg for unix and linux IO
2025-08-04 15:49:52 -03:00
PThorpe92
79629daff4
Make completions idempotent
2025-08-02 21:48:39 -04:00
PThorpe92
3048e4fa97
Add optional register_fixed_buffer method to IO trait
2025-08-01 14:54:26 -04:00
PThorpe92
ef69df7258
Apply review suggestions
2025-07-30 19:42:53 -04:00
PThorpe92
28283e4d1c
Fix bench_vfs python script to use fresh db for each run
2025-07-30 19:42:52 -04:00
PThorpe92
5f01eaae35
Fix default io:;File::pwritev impl
2025-07-30 19:42:49 -04:00
PThorpe92
7b2163208b
batch backfilling pages when checkpointing
2025-07-30 19:42:48 -04:00
PThorpe92
9c75872827
Make writing wal header able to be sync
2025-07-30 14:05:18 +03:00
PThorpe92
edbc29e2a6
Add restart_header and truncate for WAL and tests to prep for truncate checkpoint mode
2025-07-30 13:48:05 +03:00
PThorpe92
3be8bb374d
Use usize for truncate method in place of u64
2025-07-30 13:48:05 +03:00
PThorpe92
8bf2898c47
Add truncate method to IO trait and Truncate completion variant
2025-07-30 13:48:05 +03:00
pedrocarlo
d30c7d54c8
change all Arc<Completion> to Completion
2025-07-28 15:32:45 -03:00
pedrocarlo
7789c569a0
make Completion implementation contain an inner Arc<CompletionInner> so that we can must_use the Completion struct
2025-07-28 15:31:42 -03:00
Jussi Saurio
347a9152a6
Merge 'Replace verbose IO Completion methods with helpers' from Preston Thorpe
...
one of the last remnants of some original verbosity
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com >
Closes #2156
2025-07-18 10:52:17 +03:00
PThorpe92
44d7570272
Add helper/convenience methods for creating io completions
2025-07-17 23:46:55 -04:00
pedrocarlo
dc5f73887e
refactor to require Arc<Completion> in file traits so that we can delay IO calls correctly
2025-07-17 12:24:43 -03:00
Pere Diaz Bou
88ff218810
io: assert small I/O
...
Let's assert **for now** that we do not read/write less bytes than
expected. This should be fixed to retrigger several reads/writes if we
couldn't read/write enough but for now let's assert.
2025-07-14 10:19:41 +02:00
Andika Tanuwijaya
135c81f2db
ios support
2025-07-02 00:11:42 +07:00
Andika Tanuwijaya
4686592156
migrate dart binding to flutter plugin project
2025-07-02 00:04:42 +07:00
pedrocarlo
1dc28e32f0
fix io_uring completion + clippy
2025-06-26 22:17:28 -03:00
pedrocarlo
bac5e4b563
refactor File and Database Storage to remove Arc<Connection> and return Arc<Connection> for caller to wait for completion
2025-06-26 22:17:28 -03:00
pedrocarlo
64d9193e7b
refactor Completion to have a type field and lift common is_complete property
2025-06-26 22:17:27 -03:00
Pekka Enberg
59d28eac93
core: Switch Completion "is_completed" to use Cell
...
Suggested by Jussi
2025-05-27 14:05:07 +03:00
Pekka Enberg
05df548b10
core/io: Add wait_for_completion() to I/O dispatcher
2025-05-27 13:47:40 +03:00
Pekka Enberg
eca9a5b703
core/io: Switch to Arc<Completion>
2025-05-27 11:28:49 +03:00
pedrocarlo
0c22382f3c
shared lock on file and throw ReadOnly error in transaction
2025-05-02 16:30:48 -03:00
Diego Reis
d9bf383507
core/io: Untie MemoryIO's lifetime of the IO layer
2025-04-13 11:10:06 -03:00
Diego Reis
b519509349
core/io: Add internal in-memory MemoryIO to all IO layers
...
Honestly I don't have 100% sure if this is a good idea, the reasoning is that in any IO we'll want to do memory only operations like creating tables etc, so may want a common way to access it
2025-04-13 11:10:05 -03:00
Jussi Saurio
60a13c129f
io/linux: make syscallio the default (io_uring is really slow)
2025-04-10 13:32:26 +03:00
Pekka Enberg
e21e2b5449
Merge 'core: Fix syscall VFS on Linux' from Pekka Enberg
...
Fix the syscall VFS on Linux not to use `PlatformIO`, which is just an
alias for `io_uring`.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com >
Closes #1285
2025-04-09 19:38:45 +03:00
Pekka Enberg
2d009083ba
core: Fix syscall VFS on Linux
...
Fix the syscall VFS on Linux not to use `PlatformIO`, which is just an
alias for `io_uring`.
2025-04-09 19:27:58 +03:00
Avinash Sajjanshetty
2873c36b31
Implement trait Clock for implemented IOs
...
Replace `chrono::Local::now()` to return `Instant` but containing
same info
2025-04-06 23:34:13 +05:30
Avinash Sajjanshetty
8d6d50d9d5
Update IO trait
...
- Remove existing `get_current_time() -> String`
- Add super trait bound `Clock`
2025-04-06 23:06:57 +05:30
PThorpe92
57d4aa7216
Reorganize ext library and feature gate vfs to more easily prevent wasm build issues
2025-03-19 10:17:11 -04:00
PThorpe92
44f6054657
Impl copy + clone for io openflags
2025-03-12 21:52:51 -04:00
