Jussi Saurio
edf40cc65b
clippy
2025-10-09 19:00:40 +03:00
Jussi Saurio
0356a7102c
remove another expensive assert
2025-10-09 17:50:15 +03:00
Jussi Saurio
a1a83c689b
Don't yield if completion already succeeded
2025-10-09 17:50:06 +03:00
Jussi Saurio
1c35d5b342
avoid expensive Arc cloning
2025-10-09 17:43:28 +03:00
Jussi Saurio
1f310a4738
Remove expensive hot path assert
2025-10-09 17:29:18 +03:00
Jussi Saurio
bcca404551
Avoid string allocation in sorter record comparison
2025-10-09 15:34:27 +03:00
Jussi Saurio
e0461dd78a
Sorter: compute values upfront instead of deserializing on every comparison
2025-10-09 15:01:47 +03:00
Levy A.
77a412f6af
refactor: remove unsafe reference semantics from RefValue
...
also renames `RefValue` to `ValueRef`, to align with rusqlite and other
crates
2025-10-07 10:43:44 -03:00
Jussi Saurio
8dac1ba21a
Fix: actually enforce uniqueness in CREATE UNIQUE INDEX
...
...we just didn't do it
2025-10-03 22:58:42 +03:00
Pekka Enberg
7b6fc0f3b6
core/vdbe: Wrap SortedChunk::total_bytes_read with AtomicUsize
2025-09-27 14:35:31 +03:00
Pekka Enberg
61b3f56997
core/vdbe: Wrap SortedChunk::io_state with RwLock
2025-09-27 14:28:55 +03:00
Pekka Enberg
5f39987ec0
core/vdbe: Wrap SortedChunk::buffer_len with AtomicUsize
2025-09-27 14:23:02 +03:00
Pekka Enberg
b31818f77c
core/vdbe: Wrap SortedChunk::buffer with RwLock
2025-09-27 14:23:02 +03:00
PThorpe92
3c8216caab
Add Sequence and SequenceTest opcodes to vdbe and sorter
2025-09-23 22:34:13 -04:00
PThorpe92
ccae3ab0f2
Change callsites to cancel any further IO when an error occurs and drain
2025-09-08 13:18:40 -04:00
PThorpe92
0a56d23402
Use u64 for file offsets in IO and calculate such offsets in u64
2025-08-28 09:44:00 -04:00
Levy A.
4ba1304fb9
complete parser integration
2025-08-21 15:23:59 -03:00
pedrocarlo
de1811dea7
abort completions on error
2025-08-19 10:48:21 -03:00
pedrocarlo
ab3b68e360
change completion callbacks to take a Result param + create separate functions to declare a completion errored
2025-08-19 10:48:21 -03:00
pedrocarlo
d0c13f0104
remove IOError from Parser + store only ErrorKind in LimboError
2025-08-19 10:48:21 -03:00
PThorpe92
2c526c4c37
Add io_yield_x macros to reduce boilerplate
2025-08-16 16:14:00 -04:00
pedrocarlo
c381fe3844
bubble completions in Sorter
2025-08-13 10:24:55 +03:00
pedrocarlo
85e86d427b
cleanups - use io.block in many functions and return_if_io
2025-08-13 08:32:38 +03:00
pedrocarlo
f25f51b8aa
do not need wait_for_read_complete anymore
2025-08-09 21:50:18 -03:00
pedrocarlo
76d6c4a28d
only open 1 file for sorter so chunks just reuse that file
2025-08-09 21:50:18 -03:00
pedrocarlo
4a3408003a
wait for flush to complete
2025-08-09 21:50:18 -03:00
pedrocarlo
5924274d61
adjust state machine transition
2025-08-09 21:50:18 -03:00
pedrocarlo
6fe19e4ef4
adjust external sort init_chunk_heap and next_from_chunk_heap
2025-08-09 21:50:18 -03:00
pedrocarlo
c02936eb30
state machine for insert
2025-08-09 21:50:18 -03:00
pedrocarlo
2ec58b0264
state machine for sort
2025-08-09 21:50:18 -03:00
pedrocarlo
c91c22a6a8
state machine for next
2025-08-09 21:50:18 -03:00
pedrocarlo
2ffc5ee423
SorterChunk read and write should return completions
2025-08-09 21:50:18 -03:00
PThorpe92
27113885a9
Update sorter to use new buffer api
2025-08-08 10:55:26 -04:00
PThorpe92
f6a68cffc2
Remove RefCell from IO and Page apis
2025-08-05 16:24:49 -04:00
pedrocarlo
543cdb3e2c
underscoring completions and IOResult to avoid warning messages
2025-07-31 11:51:17 -03:00
pedrocarlo
3831e0db39
convert must_use compile warnings to unused_variables to track locations where we need to refactor in the future
2025-07-28 16:09:26 -03:00
pedrocarlo
d30c7d54c8
change all Arc<Completion> to Completion
2025-07-28 15:32:45 -03:00
Iaroslav Zeigerman
1e51d23bd6
store the key deserialization error instead of panicking
2025-07-23 11:22:01 -07:00
Iaroslav Zeigerman
d847c88df6
cosmetic
2025-07-22 06:37:50 +02:00
Iaroslav Zeigerman
d75e26eee4
Deserialize keys incrementally
2025-07-21 19:25:27 +02:00
Iaroslav Zeigerman
18d66b3539
Deserialize keys only once when sorting immutable records
2025-07-21 18:09:37 +02:00
Jussi Saurio
9936748132
Merge 'Avoid redundant decoding of record headers when reading sorted chunk files' from Iaroslav Zeigerman
...
Currently, each record header is decoded at least twice: once to
determine the record size within the read buffer (in order to construct
the `ImmutableRecord` instance), and again later when decoding the
record for comparison. This redundant decoding can have a noticeable
negative impact on performance when records are wide (eg. contain
multiple columns).
This update modifies the (de)serialization format for sorted chunk files
by prepending a record size varint to each record payload. As a result,
only a single varint needs to be decoded to determine the record size,
eliminating the need to decode the full record header during reads.
Closes #2176
2025-07-20 23:54:54 +03:00
Iaroslav Zeigerman
10a848fbc5
address nit
2025-07-19 18:40:43 +02:00
Iaroslav Zeigerman
5d47502e3a
Avoid redundant decoding of record headers when reading sorted chunk files
2025-07-19 06:08:27 +02:00
Jussi Saurio
4d15725b6f
sorter: fix sorter panic on SortedChunkIOState::WaitingForRead
...
The following sequence of events is possible:
- init_chunk_heap() called
- chunk A status is WriteComplete, so chunk.read() gets called on chunk A
- some other chunk B is in WaitingForWrite status after flush()
- init_chunk_heap() returns IOResult::IO
- init_chunk_heap() is called again
- we panic because chunk A is in WaitingForRead status
So - we just allow WaitingForRead status in init_chunk_heap() instead.
This panic was caught thanks to Pedro's IO latency enhancement to the sim!
2025-07-18 14:18:51 +03:00
Iaroslav Zeigerman
28ff170e14
improve sorter settings in the fuzz test
2025-07-18 07:41:15 +02:00
Iaroslav Zeigerman
76e748146b
rebase
2025-07-18 07:30:08 +02:00
Iaroslav Zeigerman
f6f1d076da
verify that records remain unchanged after sorting
2025-07-18 07:28:37 +02:00
Iaroslav Zeigerman
d9751212d7
make a fuzz sorter test
2025-07-18 07:28:37 +02:00
Iaroslav Zeigerman
20bdbd5ca5
address suggestions
2025-07-18 07:28:37 +02:00
