Commit Graph

117 Commits

Author SHA1 Message Date
PThorpe92
177c717f25 Remove windows IO in place of Generic IO 2025-08-25 18:47:21 -04:00
Pekka Enberg
5fe5e1548b core/io: Fix build on Android and iOS
Commit ebe6aa0d28 ("adjust cfg for unix
and linux IO") adjusted the I/O conditional compilation, but forgot that
Android and iOS are also part of Unix target family.

Fixes #2500
2025-08-25 11:21:46 +03:00
Nikita Sivukhin
c771487933 add remove_file method to the IO 2025-08-21 14:51:02 +04:00
Avinash Sajjanshetty
bd9b4bbfd2 encrypt/decrypt when writing/reading from DB 2025-08-20 11:47:23 +05:30
Jussi Saurio
b5439dd068 Remove assertions from Completion::complete() and Completion::error()
The completion callback can be invoked only once via `OnceLock`, let's not
crash if we e.g. call `Completion::abort()` on an already finished completion.

Closes #2673
2025-08-19 22:02:02 +03:00
pedrocarlo
66171527b4 thread safely store the result of completion 2025-08-19 10:48:21 -03:00
pedrocarlo
de1811dea7 abort completions on error 2025-08-19 10:48:21 -03:00
pedrocarlo
ab3b68e360 change completion callbacks to take a Result param + create separate functions to declare a completion errored 2025-08-19 10:48:21 -03:00
pedrocarlo
2d6fad5ea3 nit: adjust order of struct completions 2025-08-19 10:48:21 -03:00
pedrocarlo
fadf78fe67 use a dedicated Error enum for Completion Error 2025-08-19 10:48:21 -03:00
pedrocarlo
7bc0545442 default impl for get_memory_io 2025-08-19 10:48:21 -03:00
pedrocarlo
d5a59c6bee default impl for generate_random_number 2025-08-19 10:48:21 -03:00
pedrocarlo
f72bcbc5da default impl for wait_for_completion + check for errors in completion there 2025-08-19 10:48:21 -03:00
pedrocarlo
002390b5a5 store error inside Completion 2025-08-19 10:48:21 -03:00
PThorpe92
cc2fed3297 Remove copy_to API from file IO trait 2025-08-14 21:31:13 -04:00
PThorpe92
55f09a01c4 Update copy_to method in file trait to separate source and destination IO 2025-08-14 21:31:13 -04:00
Jussi Saurio
359cba0474 Use BufferPool owned by Database instead of a static global
Problem

There are several problems with our current statically allocated
`BufferPool`.

1. You cannot open two databases in the same process with different
page sizes, because the `BufferPool`'s `Arena`s will be locked forever
into the page size of the first database. This is the case regardless
of whether the two `Database`s are open at the same time, or if the first
is closed before the second is opened.

2. It is impossible to even write Rust tests for different page sizes because
of this, assuming the test uses a single process.

Solution

Make `Database` own `BufferPool` instead of it being statically allocated, so this
problem goes away.

Note that I didn't touch the still statically-allocated `TEMP_BUFFER_CACHE`, because
it should continue to work regardless of this change. It should only be a problem if
the user has two or more databases with different page sizes open simultaneously, because
`TEMP_BUFFER_CACHE` will only support one pool of a given page size at a time, so the rest
of the allocations will go through the global allocator instead.

Notes

I extracted this change out from #2569, because I didn't want it to be smuggled in without
being reviewed as an individual piece.
2025-08-14 10:29:52 +03:00
pedrocarlo
2e68296107 create IOCompletions 2025-08-13 10:24:55 +03:00
PThorpe92
213d589dd1 Apply review suggestions, remove FreeEntry 2025-08-08 11:07:29 -04:00
PThorpe92
faf248df03 Add more docs and comments for TempBufferCache 2025-08-08 10:55:28 -04:00
PThorpe92
34d90d5acb Remove Clone impl for Buffer and PageContent to make any copying of page data explicit 2025-08-08 10:55:28 -04:00
PThorpe92
d7e4ba21f8 Add explanation for using 3mb limit 2025-08-08 10:55:28 -04:00
PThorpe92
0ffba81216 Make register buffer io trait return the buf index 2025-08-08 10:55:27 -04:00
PThorpe92
cc75bc448e Move TLC buffer cache to io/mod 2025-08-08 10:55:27 -04:00
PThorpe92
a02f527c06 Add fast path for pwritev on other IO backends 2025-08-08 10:55:25 -04:00
PThorpe92
39bccc2357 Update Buffer type in io module to adjust to new pool 2025-08-08 10:55:25 -04:00
PThorpe92
bcadcb2014 Remove RefCell from copy_to method in io trait 2025-08-07 17:07:53 -04:00
PThorpe92
039fe22405 Add copy_to to io::File trait to support copying DB files 2025-08-07 16:27:02 -04:00
PThorpe92
53a0524050 Fix clippy warning 2025-08-05 16:24:50 -04:00
PThorpe92
f6a68cffc2 Remove RefCell from IO and Page apis 2025-08-05 16:24:49 -04:00
PThorpe92
914c10e095 Remove Clone impl for Buffer and PageContent 2025-08-05 14:26:53 -04:00
pedrocarlo
ebe6aa0d28 adjust cfg for unix and linux IO 2025-08-04 15:49:52 -03:00
PThorpe92
79629daff4 Make completions idempotent 2025-08-02 21:48:39 -04:00
PThorpe92
3048e4fa97 Add optional register_fixed_buffer method to IO trait 2025-08-01 14:54:26 -04:00
PThorpe92
ef69df7258 Apply review suggestions 2025-07-30 19:42:53 -04:00
PThorpe92
28283e4d1c Fix bench_vfs python script to use fresh db for each run 2025-07-30 19:42:52 -04:00
PThorpe92
5f01eaae35 Fix default io:;File::pwritev impl 2025-07-30 19:42:49 -04:00
PThorpe92
7b2163208b batch backfilling pages when checkpointing 2025-07-30 19:42:48 -04:00
PThorpe92
9c75872827 Make writing wal header able to be sync 2025-07-30 14:05:18 +03:00
PThorpe92
edbc29e2a6 Add restart_header and truncate for WAL and tests to prep for truncate checkpoint mode 2025-07-30 13:48:05 +03:00
PThorpe92
3be8bb374d Use usize for truncate method in place of u64 2025-07-30 13:48:05 +03:00
PThorpe92
8bf2898c47 Add truncate method to IO trait and Truncate completion variant 2025-07-30 13:48:05 +03:00
pedrocarlo
d30c7d54c8 change all Arc<Completion> to Completion 2025-07-28 15:32:45 -03:00
pedrocarlo
7789c569a0 make Completion implementation contain an inner Arc<CompletionInner> so that we can must_use the Completion struct 2025-07-28 15:31:42 -03:00
Jussi Saurio
347a9152a6 Merge 'Replace verbose IO Completion methods with helpers' from Preston Thorpe
one of the last remnants of some original verbosity

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2156
2025-07-18 10:52:17 +03:00
PThorpe92
44d7570272 Add helper/convenience methods for creating io completions 2025-07-17 23:46:55 -04:00
pedrocarlo
dc5f73887e refactor to require Arc<Completion> in file traits so that we can delay IO calls correctly 2025-07-17 12:24:43 -03:00
Pere Diaz Bou
88ff218810 io: assert small I/O
Let's assert **for now** that we do not read/write less bytes than
expected. This should be fixed to retrigger several reads/writes if we
couldn't read/write enough but for now let's assert.
2025-07-14 10:19:41 +02:00
Andika Tanuwijaya
135c81f2db ios support 2025-07-02 00:11:42 +07:00
Andika Tanuwijaya
4686592156 migrate dart binding to flutter plugin project 2025-07-02 00:04:42 +07:00