pedrocarlo
2ec58b0264
state machine for sort
2025-08-09 21:50:18 -03:00
pedrocarlo
c91c22a6a8
state machine for next
2025-08-09 21:50:18 -03:00
pedrocarlo
2ffc5ee423
SorterChunk read and write should return completions
2025-08-09 21:50:18 -03:00
pedrocarlo
0c9ecbc768
simulator should delete files after a successful run
2025-08-09 16:25:51 -03:00
Preston Thorpe
54613cceec
Merge 'BufferPool: add arena backed pool to support fixed opcodes and coalescing' from Preston Thorpe
...
This PR introduces a BufferPool which allocates all (usually) of the
memory we will need up front as two large arenas (one for WAL frames and
one for DB pages) and hands our pages from those two arena allocations.
(each will be 3 MB by default, because 8MB is typically the RL_MEMLOCK
limit so without `setcap cap_ipc_lock` changed, it will not be able to
properly register the arenas with io_uring).
Any additional memory needed will fall back to the previous style buffer
pool, with a simple thread local cache, which should be slightly faster
for most cases.
Closes #2419
2025-08-08 14:38:21 -04:00
PThorpe92
718114f5da
Keep free api consistent with try_alloc, size instead of pages
2025-08-08 11:50:16 -04:00
PThorpe92
213d589dd1
Apply review suggestions, remove FreeEntry
2025-08-08 11:07:29 -04:00
PThorpe92
84ffed709a
Round up allocation for wal frame arena to next page multiple of 64
2025-08-08 10:55:29 -04:00
PThorpe92
d182b836ef
Remove basic test for buffer pool to avoid race conditions in CI
2025-08-08 10:55:29 -04:00
PThorpe92
faf248df03
Add more docs and comments for TempBufferCache
2025-08-08 10:55:28 -04:00
PThorpe92
34d90d5acb
Remove Clone impl for Buffer and PageContent to make any copying of page data explicit
2025-08-08 10:55:28 -04:00
PThorpe92
d41377454c
Fix test asserting for arena size instead of page size
2025-08-08 10:55:28 -04:00
PThorpe92
d7e4ba21f8
Add explanation for using 3mb limit
2025-08-08 10:55:28 -04:00
PThorpe92
ff7d9fe961
Add basic test for buffer pool initialization and basic use
2025-08-08 10:55:28 -04:00
PThorpe92
66964fd8d2
Add documentation and comments to new buffer pool, add get_frame api
2025-08-08 10:55:28 -04:00
PThorpe92
d94e252ef9
Actually pre-register two arenas with the ring
2025-08-08 10:55:28 -04:00
PThorpe92
f9df267f42
Increase test arena size and fix import
2025-08-08 10:55:28 -04:00
PThorpe92
39d230a899
Add bitmap for tracking pages in arena
2025-08-08 10:55:27 -04:00
PThorpe92
0ffba81216
Make register buffer io trait return the buf index
2025-08-08 10:55:27 -04:00
PThorpe92
3cff47e490
Fix btree test to properly initialize pool
2025-08-08 10:55:27 -04:00
PThorpe92
fd09fe1237
Adjust io_uring to register two arenas, one for frames and the other for db pages
2025-08-08 10:55:27 -04:00
PThorpe92
cc75bc448e
Move TLC buffer cache to io/mod
2025-08-08 10:55:27 -04:00
PThorpe92
9d1ca1c8ca
Add ReadFixed/WriteFixed opcodes for buffers from registered arena
2025-08-08 10:55:27 -04:00
PThorpe92
dffa47b048
Use temp buffer for wal header
2025-08-08 10:55:27 -04:00
PThorpe92
2e072cadb0
Add CI=true env var so the tests dont each allocate 4mb by default
2025-08-08 10:55:26 -04:00
PThorpe92
7ea52a3f89
Fix changing page size and initialization for buffer pool
2025-08-08 10:55:26 -04:00
PThorpe92
036ae596c4
Fix test with old buffer api
2025-08-08 10:55:26 -04:00
PThorpe92
0884fec799
Use parent buffer pool for ephemeral pager and wal
2025-08-08 10:55:26 -04:00
PThorpe92
5750b1229c
Setup and initialize pool properly
2025-08-08 10:55:26 -04:00
PThorpe92
4ffb273b53
Adjust IO to use new buffer pool and buffer API
2025-08-08 10:55:26 -04:00
PThorpe92
27113885a9
Update sorter to use new buffer api
2025-08-08 10:55:26 -04:00
PThorpe92
a02f527c06
Add fast path for pwritev on other IO backends
2025-08-08 10:55:25 -04:00
PThorpe92
39bccc2357
Update Buffer type in io module to adjust to new pool
2025-08-08 10:55:25 -04:00
PThorpe92
d38cd6360a
Create new arena backed buffer pool
2025-08-08 10:55:25 -04:00
Pekka Enberg
81da795a66
Turso 0.1.4-pre.6
2025-08-08 16:50:24 +03:00
Pekka Enberg
6ccee4267b
Add sync package to scripts/update-version.py
2025-08-08 16:50:02 +03:00
Pekka Enberg
7e8a84a681
Merge 'turso-sync: js package' from Nikita Sivukhin
...
This PR introduces `turso-sync-js` package with the following API (see
`./packages/turso-sync-js/example/cloud/example.js`):
```js
const { connect } = require('./sync_engine.js');
const db = await connect({ path: ..., url: ..., authToken: ...});
await (await db.prepare('SELECT * FROM users')).all();
await db.exec("INSERT INTO users VALUES (12345, 'turso-sync')");
await db.sync();
```
Closes #2480
2025-08-08 16:48:18 +03:00
Preston Thorpe
d3e6172516
Merge 'global allocator should not be set for library, only for executables' from Pedro Muniz
...
We should be allocator-agnostic. It is pretty limiting for us to force a
user to use a particular allocator. This is specially restricting for
`no_std` in the future.
Reviewed-by: bit-aloo (@Shourya742)
Reviewed-by: Preston Thorpe <preston@turso.tech >
Closes #2481
2025-08-08 09:45:35 -04:00
Jussi Saurio
2fbe33d200
Merge 'translate: return parse errors for unsupported features instead of silently ignoring' from Jussi Saurio
...
Closes #1510
Closes #2505
2025-08-08 15:56:13 +03:00
Jussi Saurio
cca2f6c947
Merge 'Evaluate WHERE conditions after LEFT JOIN' from Piotr Rżysko
...
This fix ensures that `WHERE` conditions are emitted after the `LEFT
JOIN` match flag is set, so rows from the right table are properly
filtered, even when they are `NULL` due to the outer join.
Previously, the query below would return rows where `products.price` was
`NULL`:
```sql
SELECT users.id, price
FROM users
LEFT JOIN products ON users.id = products.id
WHERE products.price IS NOT NULL;
```
Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com >
Closes #2501
2025-08-08 15:54:17 +03:00
Jussi Saurio
b19a86673e
Merge 'Btree cache usable space' from Jussi Saurio
...
Built on top of #2502
```rust
/// Cached value of the usable space of a BTree page, since it is very expensive to call in a hot loop via pager.usable_space().
/// This is OK to cache because both 'PRAGMA page_size' and '.filectrl reserve_bytes' only have an effect on:
/// 1. an uninitialized database,
/// 2. an initialized database when the command is immediately followed by VACUUM.
usable_space_cached: usize,
```
Reviewed-by: Nikita Sivukhin (@sivukhin)
Closes #2503
2025-08-08 15:53:38 +03:00
Nikita Sivukhin
d2a8162e10
sync bootstrapped files in the end
2025-08-08 16:40:29 +04:00
Nikita Sivukhin
2ccdafd2a5
update sync engine wrapper
2025-08-08 16:40:19 +04:00
Nikita Sivukhin
d8847b90c6
small fixes in workflow files
2025-08-08 15:48:57 +04:00
Nikita Sivukhin
3f0162307f
configure cargo for napi-rs
2025-08-08 15:45:05 +04:00
Nikita Sivukhin
d25acf3c2c
small adjustments in the sync engine js
2025-08-08 15:44:42 +04:00
Nikita Sivukhin
ca20af6f89
fix package.json
2025-08-08 15:44:42 +04:00
Nikita Sivukhin
a289c459bc
fix CI one more time
2025-08-08 15:44:42 +04:00
Nikita Sivukhin
22858de0f0
more CI fixes
2025-08-08 15:44:42 +04:00
Nikita Sivukhin
4595d5b7a3
fix Cargo.toml
2025-08-08 15:44:42 +04:00
