pedrocarlo
543cdb3e2c
underscoring completions and IOResult to avoid warning messages
2025-07-31 11:51:17 -03:00
Jussi Saurio
7d082ab614
small fix after header accessor refactor
2025-07-31 10:05:52 +03:00
Jussi Saurio
f619556344
Merge 'Direct DatabaseHeader reads and writes – with_header and with_header_mut' from Levy A.
...
This PR introduces two methods to pager. Very much inspired by
`with_schema` and `with_schema_mut`. `Pager::with_header` and
`Pager::with_header_mut` will give to the closure a shared and unique
reference respectively that are transmuted references from the `PageRef`
buffer.
This PR also adds type-safe wrappers for `Version`, `PageSize`,
`CacheSize` and `TextEncoding`, as they have special in-memory
representations.
Writing the `DatabaseHeader` is just a single `memcpy` now.
```rs
pub fn write_database_header(&self, header: &DatabaseHeader) {
let buf = self.as_ptr();
buf[0..DatabaseHeader::SIZE].copy_from_slice(bytemuck::bytes_of(header));
}
```
`HeaderRef` and `HeaderRefMut` are used in the `with_header*` methods,
but also can be used on its own when there are multiple reads and writes
to the header, where putting everything in a closure would add too much
nesting.
Reviewed-by: Preston Thorpe (@PThorpe92)
Closes #2234
2025-07-31 10:02:47 +03:00
PThorpe92
ade1c182de
Add is_full method to checkpoint batch
2025-07-30 19:42:54 -04:00
PThorpe92
693b71449e
Clean up writev batching and apply suggestions
2025-07-30 19:42:53 -04:00
PThorpe92
ef69df7258
Apply review suggestions
2025-07-30 19:42:53 -04:00
PThorpe92
b04128b585
Fix write_pages_vectored to properly track completion
2025-07-30 19:42:50 -04:00
PThorpe92
62f004c898
Fix write counter for writev batching in checkpoint
2025-07-30 19:42:49 -04:00
PThorpe92
7b2163208b
batch backfilling pages when checkpointing
2025-07-30 19:42:48 -04:00
Levy A.
2bde1dbd42
fix: PageSize bounds check
2025-07-30 17:33:59 -03:00
Levy A.
fe66c61ff5
add usable_space to DatabaseHeader
...
we already have the `DatabaseHeader`, we don't need the cached result
2025-07-30 17:33:59 -03:00
Levy A.
e35fdb8263
feat: zero-copy DatabaseHeader
2025-07-30 17:33:59 -03:00
PThorpe92
9c75872827
Make writing wal header able to be sync
2025-07-30 14:05:18 +03:00
pedrocarlo
a1c5a6efae
begin_sync should return a Completion
2025-07-29 13:40:58 -03:00
pedrocarlo
be3793e9f1
begin_write_wal_header should return completion + wait for completion in new_shared
2025-07-29 11:45:44 -03:00
pedrocarlo
3831e0db39
convert must_use compile warnings to unused_variables to track locations where we need to refactor in the future
2025-07-28 16:09:26 -03:00
pedrocarlo
d30c7d54c8
change all Arc<Completion> to Completion
2025-07-28 15:32:45 -03:00
pedrocarlo
617254116d
begin_read_page should return completions
2025-07-28 15:31:42 -03:00
pedrocarlo
28d6245440
begin_write_btree_page should return completion
2025-07-28 15:31:42 -03:00
PThorpe92
b08c465450
Fix writing wal header for async IO
2025-07-27 21:52:13 -04:00
Glauber Costa
b8ee38868d
implement the pragma encoding
...
Do not allow setting it. That ship has sailed around 2005.
2025-07-26 19:37:39 -05:00
Pere Diaz Bou
8150a72550
check frame number is not 0
...
clippy
fmt
fix after rebase
clippy
2025-07-24 17:30:17 +02:00
Pere Diaz Bou
2ae3b3004e
ignore wal frames after bad checksum
...
SQLite basically ignores bad frames instead of panicking, let's try to
do the same.
2025-07-24 15:11:35 +02:00
Nikita Sivukhin
3d2a38eb88
add simple helper
2025-07-24 11:49:39 +04:00
Nikita Sivukhin
4a80306705
fix wal insert frame raw API
...
- we need to properly mark pages as dirty after insertion
2025-07-24 11:49:39 +04:00
Nikita Sivukhin
bf2bfbe978
fix clippy
2025-07-23 11:31:00 +04:00
Nikita Sivukhin
16763e1500
implement raw WAL write api
2025-07-23 11:30:59 +04:00
Nikita Sivukhin
d617d1d21e
fix raw read frame WAL API
2025-07-22 16:21:04 +04:00
Jussi Saurio
2967fafe73
Merge 'Usable space unwrap' from Pedro Muniz
...
Using `unwrap_or_default` can make `page_size` become 0 in this case,
which can lead to subtracting with overflow in `payload_threshold_max`
in case we have some sort of error. Better to unwrap the error here, as
in release mode we may not have overflow checks enabled to catch this.
Closes #2145
2025-07-21 00:23:06 +03:00
Iaroslav Zeigerman
5d47502e3a
Avoid redundant decoding of record headers when reading sorted chunk files
2025-07-19 06:08:27 +02:00
pedrocarlo
28ae96f49f
remove confusing casting from usize -> u16 -> usize for usable space
2025-07-18 11:36:50 -03:00
Jussi Saurio
347a9152a6
Merge 'Replace verbose IO Completion methods with helpers' from Preston Thorpe
...
one of the last remnants of some original verbosity
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com >
Closes #2156
2025-07-18 10:52:17 +03:00
Iaroslav Zeigerman
20bdbd5ca5
address suggestions
2025-07-18 07:28:37 +02:00
Iaroslav Zeigerman
78f3bf3475
Core: Introduce external sorting
2025-07-18 07:28:36 +02:00
PThorpe92
dced94aec6
Replace verbose completions with new helpers
2025-07-17 23:47:21 -04:00
pedrocarlo
c15f1e02d3
make most instrumentation levels to be Debug or Trace instead. Span creation in debug mode is very slow and impacts our ability to run the Simulator fast enough
2025-07-17 16:48:24 -03:00
pedrocarlo
7b8eec90bd
edit state machine in Btree for freeing pages + Pager state machine for free_page
2025-07-17 12:24:43 -03:00
pedrocarlo
dc5f73887e
refactor to require Arc<Completion> in file traits so that we can delay IO calls correctly
2025-07-17 12:24:43 -03:00
meteorgan
a6faab17e9
fix query page size
2025-07-15 16:34:07 +08:00
meteorgan
cf126824de
Support set page size
2025-07-15 16:34:07 +08:00
Pekka Enberg
1a0d618a41
Merge 'Assert I/O read and write sizes' from Pere Diaz Bou
...
Let's assert **for now** that we do not read/write less bytes than
expected. This should be fixed to retrigger several reads/writes if we
couldn't read/write enough but for now let's assert.
Closes #2078
2025-07-14 12:22:18 +03:00
Pere Diaz Bou
340391538a
io: change comment for assert
2025-07-14 10:36:06 +02:00
Pere Diaz Bou
88ff218810
io: assert small I/O
...
Let's assert **for now** that we do not read/write less bytes than
expected. This should be fixed to retrigger several reads/writes if we
couldn't read/write enough but for now let's assert.
2025-07-14 10:19:41 +02:00
Nikita Sivukhin
f61d733dd3
make new functions dependend on "json" Cargo feature
2025-07-14 11:26:51 +04:00
Nikita Sivukhin
c9e7271eaf
properly pass subtype
2025-07-14 11:20:49 +04:00
Nikita Sivukhin
81cd04dd65
add bin_record_json_object and table_columns_json_array functions
2025-07-14 11:19:45 +04:00
Krishna Vishal
b1f27cad94
chore: fix clippy
2025-07-14 03:28:55 +05:30
Krishna Vishal
e7e5f28c0a
chore: Clippy chill
2025-07-14 03:28:54 +05:30
Krishna Vishal
9b315d1d7e
Manually inline the record deserialization code for performance.
...
This is done because the compiler is refusing to inline even after
adding inline hint.
- Get refvalues from directly from registers without using
`make_record`
2025-07-14 03:28:54 +05:30
Krishna Vishal
860de412d9
Add num_columns to BTreeCursor so we can initialize
...
`Vecs` inside `RecordCursor` to their appropriate to reduce
allocations.
2025-07-14 03:28:54 +05:30
