Commit Graph

4083 Commits

Author SHA1 Message Date
Lucas Forato
4161f68774 Update core/lib.rs
Co-authored-by: bit-aloo <84662239+Shourya742@users.noreply.github.com>
2025-08-11 08:42:03 -03:00
Lucas Forato
6b1ae89426 chore: remove todo comment 2025-08-11 08:42:03 -03:00
Lucas Forato
5aa99f6743 feat: listing all modules 2025-08-11 08:42:03 -03:00
Pekka Enberg
cdaea7f274 core/vdbe: Make apply_view_deltas() return early if views are disabled
Currently, we have a borrow problem because parse_schema_rows() already
borrows `schema`, but then `apply_view_deltas` does the same:

```
thread 'main' panicked at core/vdbe/mod.rs:450:49:
already mutably borrowed: BorrowError
stack backtrace:
   0: __rustc::rust_begin_unwind
             at /rustc/6b00bc3880198600130e1cf62b8f8a93494488cc/library/std/src/panicking.rs:697:5
   1: core::panicking::panic_fmt
             at /rustc/6b00bc3880198600130e1cf62b8f8a93494488cc/library/core/src/panicking.rs:75:14
   2: core::cell::panic_already_mutably_borrowed
             at /rustc/6b00bc3880198600130e1cf62b8f8a93494488cc/library/core/src/cell.rs:799:5
   3: core::cell::RefCell<T>::borrow
             at /rustc/6b00bc3880198600130e1cf62b8f8a93494488cc/library/core/src/cell.rs:987:25
   4: turso_core::vdbe::Program::apply_view_deltas
             at ./core/vdbe/mod.rs:450:26
   5: turso_core::vdbe::Program::commit_txn
             at ./core/vdbe/mod.rs:468:9
   6: turso_core::vdbe::execute::op_halt
             at ./core/vdbe/execute.rs:1954:15
   7: turso_core::vdbe::Program::step
             at ./core/vdbe/mod.rs:430:19
   8: turso_core::Statement::step
             at ./core/lib.rs:1914:23
   9: turso_core::util::parse_schema_rows
             at ./core/util.rs:91:15
  10: turso_core::Connection::parse_schema_rows::{{closure}}
             at ./core/lib.rs:1518:17
  11: turso_core::Connection::with_schema_mut
             at ./core/lib.rs:1625:9
  12: turso_core::Connection::parse_schema_rows
             at ./core/lib.rs:1515:9
```

However, this is a read transaction and views are not even enabled,
let's just make `apply_view_deltas()` return early if there's no
processing needed, to skip the schema borrow altogether.
2025-08-11 12:26:11 +03:00
Pekka Enberg
3aa885690c core/vdbe: Don't call rowid() unless view processing is enabled
Currently, the simulator complains of the following error:

```
Error: failed with error: 'attempt to multiply with overflow'
```

However, we don't enable views in the simulator so -- despite being an
issue -- we should never see this. Let's fix `op_delete()` some more not
to not even call rowid() unless view processing is enabled.
2025-08-11 10:56:58 +03:00
Pekka Enberg
6f3278f6ed core/vdbe: Perform view processing in op_delete only if views are enabled
Like in op_insert(), let's perform view processing only if views are
actually enabled.
2025-08-11 10:26:16 +03:00
Pekka Enberg
62f1fd2038 core/incremental: Make clippy happy 2025-08-11 08:36:53 +03:00
Pekka Enberg
87322ad1e4 core/incremental: Evaluate view expressions
...tests were failing because we are testing with expressions, but
didn't support them.
2025-08-11 08:27:10 +03:00
Pekka Enberg
6e8d1a5c7d Merge 'Initial pass on incremental view maintenance with DBSP' from Glauber Costa
Implement very basic views using DBSP
This is just the bare minimum that I needed to convince myself that this
 approach will work. The only views that we support are slices of the
 main table: no aggregations, no joins, no projections.
 * drop view is implemented.
 * view population is implemented.
 * deletes, inserts and updates are implemented.
 much like indexes before, a flag must be passed to enable views.

Closes #2530
2025-08-11 07:45:43 +03:00
Glauber Costa
145d6eede7 Implement very basic views using DBSP
This is just the bare minimum that I needed to convince myself that this
approach will work. The only views that we support are slices of the
main table: no aggregations, no joins, no projections.

drop view is implemented.
view population is implemented.
deletes, inserts and updates are implemented.

much like indexes before, a flag must be passed to enable views.
2025-08-10 23:34:04 -05:00
Glauber Costa
d5b7533ff8 Implement a DBSP module
We are not using the DBSP crate because it is very heavy on Tokio and
other dependencies that won't make sense for us to consume.
2025-08-10 23:15:26 -05:00
Glauber Costa
e255fc9a81 Add table name to the delete bytecode
When building views (soon), it will be important to know which table
is being deleted. Getting from the cursor id is very cumbersome.

What we are doing here is symmetrical to op_insert, and sqlite also
passes table information in one of the registers (p4)
2025-08-10 22:50:25 -05:00
PThorpe92
01f281bae5 Fix doc test/comment 2025-08-09 14:41:15 -04:00
PThorpe92
3d518baef5 Fix assertion to make sure a reader or writer exists when unlock is called 2025-08-09 14:40:03 -04:00
PThorpe92
f81bc3236a Use a single packed u64 for LimboRwLock and relax memory ordering 2025-08-09 14:08:25 -04:00
PThorpe92
718114f5da Keep free api consistent with try_alloc, size instead of pages 2025-08-08 11:50:16 -04:00
PThorpe92
213d589dd1 Apply review suggestions, remove FreeEntry 2025-08-08 11:07:29 -04:00
PThorpe92
84ffed709a Round up allocation for wal frame arena to next page multiple of 64 2025-08-08 10:55:29 -04:00
PThorpe92
d182b836ef Remove basic test for buffer pool to avoid race conditions in CI 2025-08-08 10:55:29 -04:00
PThorpe92
faf248df03 Add more docs and comments for TempBufferCache 2025-08-08 10:55:28 -04:00
PThorpe92
34d90d5acb Remove Clone impl for Buffer and PageContent to make any copying of page data explicit 2025-08-08 10:55:28 -04:00
PThorpe92
d41377454c Fix test asserting for arena size instead of page size 2025-08-08 10:55:28 -04:00
PThorpe92
d7e4ba21f8 Add explanation for using 3mb limit 2025-08-08 10:55:28 -04:00
PThorpe92
ff7d9fe961 Add basic test for buffer pool initialization and basic use 2025-08-08 10:55:28 -04:00
PThorpe92
66964fd8d2 Add documentation and comments to new buffer pool, add get_frame api 2025-08-08 10:55:28 -04:00
PThorpe92
d94e252ef9 Actually pre-register two arenas with the ring 2025-08-08 10:55:28 -04:00
PThorpe92
f9df267f42 Increase test arena size and fix import 2025-08-08 10:55:28 -04:00
PThorpe92
39d230a899 Add bitmap for tracking pages in arena 2025-08-08 10:55:27 -04:00
PThorpe92
0ffba81216 Make register buffer io trait return the buf index 2025-08-08 10:55:27 -04:00
PThorpe92
3cff47e490 Fix btree test to properly initialize pool 2025-08-08 10:55:27 -04:00
PThorpe92
fd09fe1237 Adjust io_uring to register two arenas, one for frames and the other for db pages 2025-08-08 10:55:27 -04:00
PThorpe92
cc75bc448e Move TLC buffer cache to io/mod 2025-08-08 10:55:27 -04:00
PThorpe92
9d1ca1c8ca Add ReadFixed/WriteFixed opcodes for buffers from registered arena 2025-08-08 10:55:27 -04:00
PThorpe92
dffa47b048 Use temp buffer for wal header 2025-08-08 10:55:27 -04:00
PThorpe92
7ea52a3f89 Fix changing page size and initialization for buffer pool 2025-08-08 10:55:26 -04:00
PThorpe92
0884fec799 Use parent buffer pool for ephemeral pager and wal 2025-08-08 10:55:26 -04:00
PThorpe92
5750b1229c Setup and initialize pool properly 2025-08-08 10:55:26 -04:00
PThorpe92
4ffb273b53 Adjust IO to use new buffer pool and buffer API 2025-08-08 10:55:26 -04:00
PThorpe92
27113885a9 Update sorter to use new buffer api 2025-08-08 10:55:26 -04:00
PThorpe92
a02f527c06 Add fast path for pwritev on other IO backends 2025-08-08 10:55:25 -04:00
PThorpe92
39bccc2357 Update Buffer type in io module to adjust to new pool 2025-08-08 10:55:25 -04:00
PThorpe92
d38cd6360a Create new arena backed buffer pool 2025-08-08 10:55:25 -04:00
Preston Thorpe
d3e6172516 Merge 'global allocator should not be set for library, only for executables' from Pedro Muniz
We should be allocator-agnostic. It is pretty limiting for us to force a
user to use a particular allocator. This is specially restricting for
`no_std` in the future.

Reviewed-by: bit-aloo (@Shourya742)
Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2481
2025-08-08 09:45:35 -04:00
Jussi Saurio
2fbe33d200 Merge 'translate: return parse errors for unsupported features instead of silently ignoring' from Jussi Saurio
Closes #1510

Closes #2505
2025-08-08 15:56:13 +03:00
Jussi Saurio
cca2f6c947 Merge 'Evaluate WHERE conditions after LEFT JOIN' from Piotr Rżysko
This fix ensures that `WHERE` conditions are emitted after the `LEFT
JOIN` match flag is set, so rows from the right table are properly
filtered, even when they are `NULL` due to the outer join.
Previously, the query below would return rows where `products.price` was
`NULL`:
```sql
SELECT users.id, price
FROM users
LEFT JOIN products ON users.id = products.id
WHERE products.price IS NOT NULL;
```

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2501
2025-08-08 15:54:17 +03:00
Jussi Saurio
21dc2d0161 translate: return parse errors for unsupported features instead of silently ignoring 2025-08-08 11:39:30 +03:00
Jussi Saurio
7fd63d8a5d btree: cache usable_space in the btreecursor constructor 2025-08-08 10:32:18 +03:00
Jussi Saurio
15c429b673 btree: remove completely unused ParseRecordState 2025-08-08 10:08:59 +03:00
Pekka Enberg
0f9d0cf519 Merge branch 'main' into 2025-08-07-add-query-only-pragma 2025-08-08 07:41:38 +03:00
Piotr Rzysko
375b9047e2 Evaluate WHERE conditions after LEFT JOIN
Previously, the query from the added test would not filter out rows
where `products.price` was NULL.
2025-08-08 06:26:30 +02:00