Pekka Enberg
d959319b42
Merge 'Use u64 for file offsets in I/O and calculate such offsets in u64' from Preston Thorpe
...
Using `usize` to compute file offsets caps us at ~16GB on 32-bit
systems. For example, with 4 KiB pages we can only address up to 1048576
pages; attempting the next page overflows a 32-bit usize and can wrap
the write offset, corrupting data. Switching our I/O APIs and offset
math to u64 avoids this overflow on 32-bit targets
Closes #2791
2025-09-02 09:06:49 +03:00
Avinash Sajjanshetty
bb591ab7e1
Propagate decryption erorr when reading from WAL
2025-08-29 18:07:38 +05:30
Avinash Sajjanshetty
95d14f5f1f
Propagate decryption error from the callback
2025-08-29 18:02:56 +05:30
PThorpe92
fb7e6fb280
Guard against all overflow when calculating IO offsets
2025-08-28 09:50:26 -04:00
PThorpe92
0a56d23402
Use u64 for file offsets in IO and calculate such offsets in u64
2025-08-28 09:44:00 -04:00
Avinash Sajjanshetty
2c0842ff52
Set and propagate IOContext as required
2025-08-27 22:05:01 +05:30
Avinash Sajjanshetty
9e663c7f46
Add IOContext to carry encryption/checksum ctx
2025-08-27 21:33:05 +05:30
Pekka Enberg
2c6fa76437
cargo fmt
2025-08-24 14:13:20 +03:00
Pekka Enberg
22c9cb6618
s/PerConnEncryptionContext/EncryptionContext/
2025-08-24 08:17:20 +03:00
Avinash Sajjanshetty
3090545167
use encryption ctx instead of encryption key
2025-08-21 22:36:32 +05:30
Avinash Sajjanshetty
40a209c000
simplify feature flag usage for encryption
2025-08-20 12:49:38 +05:30
Avinash Sajjanshetty
bd9b4bbfd2
encrypt/decrypt when writing/reading from DB
2025-08-20 11:47:23 +05:30
Avinash Sajjanshetty
201262b3dd
Update DatabaseStorage to pass encryption context
2025-08-20 11:41:08 +05:30
PThorpe92
cc2fed3297
Remove copy_to API from file IO trait
2025-08-14 21:31:13 -04:00
PThorpe92
3c088dda59
Update callsites of copy_to Database impl
2025-08-14 21:31:13 -04:00
Jussi Saurio
c75e4c1092
Fix non-4096 page sizes by making WAL header lazy
2025-08-14 12:40:58 +03:00
Jussi Saurio
ee58b7bd86
Add fn read_header() to DatabaseStorage trait
2025-08-14 12:40:58 +03:00
PThorpe92
039fe22405
Add copy_to to io::File trait to support copying DB files
2025-08-07 16:27:02 -04:00
PThorpe92
f6a68cffc2
Remove RefCell from IO and Page apis
2025-08-05 16:24:49 -04:00
Jussi Saurio
456b7404fb
storage: remove FileMemoryStorage as it is never used
2025-08-01 10:14:36 +03:00
PThorpe92
ef69df7258
Apply review suggestions
2025-07-30 19:42:53 -04:00
PThorpe92
7b2163208b
batch backfilling pages when checkpointing
2025-07-30 19:42:48 -04:00
PThorpe92
2c3a9fe5ef
Finish wal transaction handling and add more wal and chkpt testing
2025-07-30 14:10:10 +03:00
PThorpe92
9c75872827
Make writing wal header able to be sync
2025-07-30 14:05:18 +03:00
PThorpe92
dcf346b9b0
Update apparently very old branch to new api
2025-07-30 13:48:05 +03:00
PThorpe92
272a63b562
Fix truncate impl in JS bindings
2025-07-30 13:48:05 +03:00
PThorpe92
73fe4ffa06
Add truncate method to DatabaseStorage trait
2025-07-30 13:48:05 +03:00
pedrocarlo
3831e0db39
convert must_use compile warnings to unused_variables to track locations where we need to refactor in the future
2025-07-28 16:09:26 -03:00
pedrocarlo
d30c7d54c8
change all Arc<Completion> to Completion
2025-07-28 15:32:45 -03:00
pedrocarlo
7789c569a0
make Completion implementation contain an inner Arc<CompletionInner> so that we can must_use the Completion struct
2025-07-28 15:31:42 -03:00
pedrocarlo
3104e3fee5
adjust DatabaseStorage trait to return completions
2025-07-28 15:31:42 -03:00
pedrocarlo
c15f1e02d3
make most instrumentation levels to be Debug or Trace instead. Span creation in debug mode is very slow and impacts our ability to run the Simulator fast enough
2025-07-17 16:48:24 -03:00
pedrocarlo
dc5f73887e
refactor to require Arc<Completion> in file traits so that we can delay IO calls correctly
2025-07-17 12:24:43 -03:00
pedrocarlo
b85687658d
change instrumentation level to INFO
2025-07-07 11:53:45 -03:00
pedrocarlo
5559c45011
more instrumentation + write counter should decrement if pwrite fails
2025-07-07 11:50:21 -03:00
pedrocarlo
1dc28e32f0
fix io_uring completion + clippy
2025-06-26 22:17:28 -03:00
pedrocarlo
bac5e4b563
refactor File and Database Storage to remove Arc<Connection> and return Arc<Connection> for caller to wait for completion
2025-06-26 22:17:28 -03:00
pedrocarlo
64d9193e7b
refactor Completion to have a type field and lift common is_complete property
2025-06-26 22:17:27 -03:00
Diego Reis
9c7330c01c
core: Add size method to DatabaseStorage trait
2025-06-24 14:41:49 -03:00
Pekka Enberg
eca9a5b703
core/io: Switch to Arc<Completion>
2025-05-27 11:28:49 +03:00
Diego Reis
79f8b83cbe
Fix dumb clippy errors
2025-04-13 11:10:06 -03:00
Diego Reis
e5144bb6a9
core/storage: Create FileMemoryStorage
...
This is basically a copy of DatabaseStorage but outside the fs compilation flag, this way, we can access MemoryIO regardless the storage medium.
2025-04-13 11:10:06 -03:00
TcMits
56fa9049c3
fix: overflow pos in write_page
2025-04-03 15:02:53 +07:00
Pekka Enberg
df6af6ed79
core: Rename FileStorage to DatabaseFile
2025-03-25 11:15:16 +02:00
PThorpe92
e22c86c230
Impl Copy on some types in the pager to prevent explicit clones
2025-03-12 21:57:35 -04:00
Pere Diaz Bou
3d3cdbeb0c
fix generic and windows sync compilation
2025-03-05 14:07:48 +01:00
Pere Diaz Bou
e4a8ee5402
move load extensions to Connection
...
Extensions are loaded per connection and not per database as per SQLite
behaviour. This also helps with removing locks.
2025-03-05 14:07:48 +01:00
Pere Diaz Bou
8daf7666d1
Make database Sync + Send
2025-03-05 14:07:48 +01:00
Pekka Enberg
0638550be7
Merge 'Remove unnecessary reference counting from completion I/O callbacks' from Preston Thorpe
...
I am on a bit of a mission to revisit a lot of the ref counting, this
was an easy first win.
It seems to be a linear path of function calls or hashmaps which can own
the completions directly, no cloning needed.
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com >
Closes #912
2025-02-10 12:11:30 +02:00
Jorge López
83b158fb3a
core: silence some unused warnings when building without default features
2025-02-09 01:13:12 +01:00
