Jussi Saurio
4b27cc0d46
btree: add fast path version of cell_get_raw_region
2025-08-07 09:57:56 +03:00
Jussi Saurio
c98136c8c4
btree: use new cell start helper method in cell_get_raw_region
2025-08-07 09:37:33 +03:00
Jussi Saurio
3db25cf84c
perf/btree: add method for getting raw offset of cell payload start
2025-08-07 09:34:05 +03:00
PThorpe92
f6a68cffc2
Remove RefCell from IO and Page apis
2025-08-05 16:24:49 -04:00
PThorpe92
914c10e095
Remove Clone impl for Buffer and PageContent
2025-08-05 14:26:53 -04:00
Jussi Saurio
a28e64bfdd
cleanup: remove unused page uptodate flag
2025-08-05 14:25:42 +03:00
Jussi Saurio
5b84ad6b0f
Merge 'Update defragment page to defragment in-place' from João Severo
...
Change original code from doing a full copy of the original buffer to
modify the buffer in-place using a temporary vector with offsets.
Closes #2258
2025-08-05 11:22:22 +03:00
Nikita Sivukhin
2e23230e79
extend raw WAL API with few more methods
...
- try_wal_watermark_read_page - try to read page from the DB with given WAL watermark value
- wal_changed_pages_after - return set of unique pages changed after watermark WAL position
2025-08-04 16:55:50 +04:00
Joao Severo
785c3fdb03
Split into two methods
2025-08-02 13:24:12 +03:00
pedrocarlo
543cdb3e2c
underscoring completions and IOResult to avoid warning messages
2025-07-31 11:51:17 -03:00
Jussi Saurio
7d082ab614
small fix after header accessor refactor
2025-07-31 10:05:52 +03:00
Jussi Saurio
f619556344
Merge 'Direct DatabaseHeader reads and writes – with_header and with_header_mut' from Levy A.
...
This PR introduces two methods to pager. Very much inspired by
`with_schema` and `with_schema_mut`. `Pager::with_header` and
`Pager::with_header_mut` will give to the closure a shared and unique
reference respectively that are transmuted references from the `PageRef`
buffer.
This PR also adds type-safe wrappers for `Version`, `PageSize`,
`CacheSize` and `TextEncoding`, as they have special in-memory
representations.
Writing the `DatabaseHeader` is just a single `memcpy` now.
```rs
pub fn write_database_header(&self, header: &DatabaseHeader) {
let buf = self.as_ptr();
buf[0..DatabaseHeader::SIZE].copy_from_slice(bytemuck::bytes_of(header));
}
```
`HeaderRef` and `HeaderRefMut` are used in the `with_header*` methods,
but also can be used on its own when there are multiple reads and writes
to the header, where putting everything in a closure would add too much
nesting.
Reviewed-by: Preston Thorpe (@PThorpe92)
Closes #2234
2025-07-31 10:02:47 +03:00
PThorpe92
ade1c182de
Add is_full method to checkpoint batch
2025-07-30 19:42:54 -04:00
PThorpe92
693b71449e
Clean up writev batching and apply suggestions
2025-07-30 19:42:53 -04:00
PThorpe92
ef69df7258
Apply review suggestions
2025-07-30 19:42:53 -04:00
PThorpe92
b04128b585
Fix write_pages_vectored to properly track completion
2025-07-30 19:42:50 -04:00
PThorpe92
62f004c898
Fix write counter for writev batching in checkpoint
2025-07-30 19:42:49 -04:00
PThorpe92
7b2163208b
batch backfilling pages when checkpointing
2025-07-30 19:42:48 -04:00
Levy A.
2bde1dbd42
fix: PageSize bounds check
2025-07-30 17:33:59 -03:00
Levy A.
fe66c61ff5
add usable_space to DatabaseHeader
...
we already have the `DatabaseHeader`, we don't need the cached result
2025-07-30 17:33:59 -03:00
Levy A.
e35fdb8263
feat: zero-copy DatabaseHeader
2025-07-30 17:33:59 -03:00
PThorpe92
9c75872827
Make writing wal header able to be sync
2025-07-30 14:05:18 +03:00
pedrocarlo
a1c5a6efae
begin_sync should return a Completion
2025-07-29 13:40:58 -03:00
pedrocarlo
be3793e9f1
begin_write_wal_header should return completion + wait for completion in new_shared
2025-07-29 11:45:44 -03:00
pedrocarlo
3831e0db39
convert must_use compile warnings to unused_variables to track locations where we need to refactor in the future
2025-07-28 16:09:26 -03:00
pedrocarlo
d30c7d54c8
change all Arc<Completion> to Completion
2025-07-28 15:32:45 -03:00
pedrocarlo
617254116d
begin_read_page should return completions
2025-07-28 15:31:42 -03:00
pedrocarlo
28d6245440
begin_write_btree_page should return completion
2025-07-28 15:31:42 -03:00
PThorpe92
b08c465450
Fix writing wal header for async IO
2025-07-27 21:52:13 -04:00
Glauber Costa
b8ee38868d
implement the pragma encoding
...
Do not allow setting it. That ship has sailed around 2005.
2025-07-26 19:37:39 -05:00
Pere Diaz Bou
8150a72550
check frame number is not 0
...
clippy
fmt
fix after rebase
clippy
2025-07-24 17:30:17 +02:00
Pere Diaz Bou
2ae3b3004e
ignore wal frames after bad checksum
...
SQLite basically ignores bad frames instead of panicking, let's try to
do the same.
2025-07-24 15:11:35 +02:00
Nikita Sivukhin
3d2a38eb88
add simple helper
2025-07-24 11:49:39 +04:00
Nikita Sivukhin
4a80306705
fix wal insert frame raw API
...
- we need to properly mark pages as dirty after insertion
2025-07-24 11:49:39 +04:00
Nikita Sivukhin
bf2bfbe978
fix clippy
2025-07-23 11:31:00 +04:00
Nikita Sivukhin
16763e1500
implement raw WAL write api
2025-07-23 11:30:59 +04:00
Nikita Sivukhin
d617d1d21e
fix raw read frame WAL API
2025-07-22 16:21:04 +04:00
Jussi Saurio
2967fafe73
Merge 'Usable space unwrap' from Pedro Muniz
...
Using `unwrap_or_default` can make `page_size` become 0 in this case,
which can lead to subtracting with overflow in `payload_threshold_max`
in case we have some sort of error. Better to unwrap the error here, as
in release mode we may not have overflow checks enabled to catch this.
Closes #2145
2025-07-21 00:23:06 +03:00
Iaroslav Zeigerman
5d47502e3a
Avoid redundant decoding of record headers when reading sorted chunk files
2025-07-19 06:08:27 +02:00
pedrocarlo
28ae96f49f
remove confusing casting from usize -> u16 -> usize for usable space
2025-07-18 11:36:50 -03:00
Jussi Saurio
347a9152a6
Merge 'Replace verbose IO Completion methods with helpers' from Preston Thorpe
...
one of the last remnants of some original verbosity
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com >
Closes #2156
2025-07-18 10:52:17 +03:00
Iaroslav Zeigerman
20bdbd5ca5
address suggestions
2025-07-18 07:28:37 +02:00
Iaroslav Zeigerman
78f3bf3475
Core: Introduce external sorting
2025-07-18 07:28:36 +02:00
PThorpe92
dced94aec6
Replace verbose completions with new helpers
2025-07-17 23:47:21 -04:00
pedrocarlo
c15f1e02d3
make most instrumentation levels to be Debug or Trace instead. Span creation in debug mode is very slow and impacts our ability to run the Simulator fast enough
2025-07-17 16:48:24 -03:00
pedrocarlo
7b8eec90bd
edit state machine in Btree for freeing pages + Pager state machine for free_page
2025-07-17 12:24:43 -03:00
pedrocarlo
dc5f73887e
refactor to require Arc<Completion> in file traits so that we can delay IO calls correctly
2025-07-17 12:24:43 -03:00
meteorgan
a6faab17e9
fix query page size
2025-07-15 16:34:07 +08:00
meteorgan
cf126824de
Support set page size
2025-07-15 16:34:07 +08:00
Pekka Enberg
1a0d618a41
Merge 'Assert I/O read and write sizes' from Pere Diaz Bou
...
Let's assert **for now** that we do not read/write less bytes than
expected. This should be fixed to retrigger several reads/writes if we
couldn't read/write enough but for now let's assert.
Closes #2078
2025-07-14 12:22:18 +03:00
