Commit Graph

4116 Commits

Author SHA1 Message Date
Jussi Saurio
f598c86fa4 Merge 'Handle single, double and unquoted strings in values clause' from Mikaël Francoeur
I'm not sure how much this will clash with @TcMits's parser rewrite,
hopefully not too much. If it does and we eventually have to remove it,
at least we'll have two new regression tests.
Closes https://github.com/tursodatabase/turso/issues/2484

Closes #2499
2025-08-11 21:08:15 +03:00
Jussi Saurio
a7c1c8b7d4 Merge 'Use rusqlite 0.37 with bundled SQLite everywhere' from Jussi Saurio
Use the same rusqlite version in every crate and use a bundled up-to-
date sqlite version
(the impetus for this PR is still me trying to figure out why sqlite in
the insert benchmark doesn't seem to be fsyncing, even when instructed)

Closes #2507
2025-08-11 21:07:24 +03:00
Preston Thorpe
cb59877821 Merge 'Feat/pragma module list' from Lucas Forato
Implementation of the module_list pragma

Reviewed-by: Diego Reis (@el-yawd)
Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2430
2025-08-11 13:54:24 -04:00
PThorpe92
4dbf69545e try_borrow.unwrap -> borrow 2025-08-11 13:54:00 -04:00
Lucas Forato
e07ab423e6 feat: removed module_name from VirtualTable, instead looking only at symbol tables 2025-08-11 11:43:32 -03:00
Pekka Enberg
9a35d76c55 Merge 'Add support for Full checkpoint mode in the WAL' from Preston Thorpe
Closes #2519

Closes #2521
2025-08-11 16:59:26 +03:00
Jussi Saurio
ec7bded092 Merge 'Sorter IO Completions' from Pedro Muniz
Convert Sorter code to use state machines and not ignore completions.
Also simplifies some logic that seemed redundant to me. Also, I was
getting some IO errors because we were opening one file per Chunk, so I
fixed this by using only one file per sorter, and just using offsets in
the file for each chunk.
Builds on top of #2520

Closes #2473
2025-08-11 15:42:58 +03:00
Jussi Saurio
87bf488bbc chore: use rusqlite 0.37 with bundled sqlite everywhere 2025-08-11 15:13:57 +03:00
Pekka Enberg
221e030d98 Merge 'Fix performance regression in prepare' from Piotr Rżysko
Attempt at fixing https://github.com/tursodatabase/turso/issues/2527.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2532
2025-08-11 15:08:10 +03:00
Pekka Enberg
4a53c44466 Merge 'Add support for PRAGMA freelist_count' from bit-aloo
Implements `PRAGMA freelist_count` to return the current number of free
pages in the database.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2531
2025-08-11 15:07:58 +03:00
Lucas Forato
9fe20a178a fix: formatting 2025-08-11 09:01:27 -03:00
Lucas Forato
76838bb46a fix: issues from rebase 2025-08-11 08:54:42 -03:00
Lucas Forato
562076fdb8 fix: close method 2025-08-11 08:42:19 -03:00
Lucas Forato
19c445f8b1 feat: listing both used and non-used modules 2025-08-11 08:42:08 -03:00
Lucas Forato
adb69f21f9 feat: remove mods 2025-08-11 08:42:08 -03:00
Lucas Forato
8b1b79bc93 fix: broken file after merge 2025-08-11 08:42:08 -03:00
Lucas Forato
4161f68774 Update core/lib.rs
Co-authored-by: bit-aloo <84662239+Shourya742@users.noreply.github.com>
2025-08-11 08:42:03 -03:00
Lucas Forato
6b1ae89426 chore: remove todo comment 2025-08-11 08:42:03 -03:00
Lucas Forato
5aa99f6743 feat: listing all modules 2025-08-11 08:42:03 -03:00
Pekka Enberg
cdaea7f274 core/vdbe: Make apply_view_deltas() return early if views are disabled
Currently, we have a borrow problem because parse_schema_rows() already
borrows `schema`, but then `apply_view_deltas` does the same:

```
thread 'main' panicked at core/vdbe/mod.rs:450:49:
already mutably borrowed: BorrowError
stack backtrace:
   0: __rustc::rust_begin_unwind
             at /rustc/6b00bc3880198600130e1cf62b8f8a93494488cc/library/std/src/panicking.rs:697:5
   1: core::panicking::panic_fmt
             at /rustc/6b00bc3880198600130e1cf62b8f8a93494488cc/library/core/src/panicking.rs:75:14
   2: core::cell::panic_already_mutably_borrowed
             at /rustc/6b00bc3880198600130e1cf62b8f8a93494488cc/library/core/src/cell.rs:799:5
   3: core::cell::RefCell<T>::borrow
             at /rustc/6b00bc3880198600130e1cf62b8f8a93494488cc/library/core/src/cell.rs:987:25
   4: turso_core::vdbe::Program::apply_view_deltas
             at ./core/vdbe/mod.rs:450:26
   5: turso_core::vdbe::Program::commit_txn
             at ./core/vdbe/mod.rs:468:9
   6: turso_core::vdbe::execute::op_halt
             at ./core/vdbe/execute.rs:1954:15
   7: turso_core::vdbe::Program::step
             at ./core/vdbe/mod.rs:430:19
   8: turso_core::Statement::step
             at ./core/lib.rs:1914:23
   9: turso_core::util::parse_schema_rows
             at ./core/util.rs:91:15
  10: turso_core::Connection::parse_schema_rows::{{closure}}
             at ./core/lib.rs:1518:17
  11: turso_core::Connection::with_schema_mut
             at ./core/lib.rs:1625:9
  12: turso_core::Connection::parse_schema_rows
             at ./core/lib.rs:1515:9
```

However, this is a read transaction and views are not even enabled,
let's just make `apply_view_deltas()` return early if there's no
processing needed, to skip the schema borrow altogether.
2025-08-11 12:26:11 +03:00
Pekka Enberg
3aa885690c core/vdbe: Don't call rowid() unless view processing is enabled
Currently, the simulator complains of the following error:

```
Error: failed with error: 'attempt to multiply with overflow'
```

However, we don't enable views in the simulator so -- despite being an
issue -- we should never see this. Let's fix `op_delete()` some more not
to not even call rowid() unless view processing is enabled.
2025-08-11 10:56:58 +03:00
Pekka Enberg
6f3278f6ed core/vdbe: Perform view processing in op_delete only if views are enabled
Like in op_insert(), let's perform view processing only if views are
actually enabled.
2025-08-11 10:26:16 +03:00
Piotr Rzysko
40a8058083 Use mimalloc in benchmark.rs 2025-08-11 08:02:35 +02:00
Pekka Enberg
62f1fd2038 core/incremental: Make clippy happy 2025-08-11 08:36:53 +03:00
Pekka Enberg
87322ad1e4 core/incremental: Evaluate view expressions
...tests were failing because we are testing with expressions, but
didn't support them.
2025-08-11 08:27:10 +03:00
Pekka Enberg
6e8d1a5c7d Merge 'Initial pass on incremental view maintenance with DBSP' from Glauber Costa
Implement very basic views using DBSP
This is just the bare minimum that I needed to convince myself that this
 approach will work. The only views that we support are slices of the
 main table: no aggregations, no joins, no projections.
 * drop view is implemented.
 * view population is implemented.
 * deletes, inserts and updates are implemented.
 much like indexes before, a flag must be passed to enable views.

Closes #2530
2025-08-11 07:45:43 +03:00
Glauber Costa
145d6eede7 Implement very basic views using DBSP
This is just the bare minimum that I needed to convince myself that this
approach will work. The only views that we support are slices of the
main table: no aggregations, no joins, no projections.

drop view is implemented.
view population is implemented.
deletes, inserts and updates are implemented.

much like indexes before, a flag must be passed to enable views.
2025-08-10 23:34:04 -05:00
bit-aloo
d2171e24a5 add pragma freelist_count 2025-08-11 10:03:46 +05:30
bit-aloo
cf12c90428 expose freepage_list in pager 2025-08-11 09:57:46 +05:30
Glauber Costa
d5b7533ff8 Implement a DBSP module
We are not using the DBSP crate because it is very heavy on Tokio and
other dependencies that won't make sense for us to consume.
2025-08-10 23:15:26 -05:00
Glauber Costa
e255fc9a81 Add table name to the delete bytecode
When building views (soon), it will be important to know which table
is being deleted. Getting from the cursor id is very cumbersome.

What we are doing here is symmetrical to op_insert, and sqlite also
passes table information in one of the registers (p4)
2025-08-10 22:50:25 -05:00
pedrocarlo
f25f51b8aa do not need wait_for_read_complete anymore 2025-08-09 21:50:18 -03:00
pedrocarlo
76d6c4a28d only open 1 file for sorter so chunks just reuse that file 2025-08-09 21:50:18 -03:00
pedrocarlo
4a3408003a wait for flush to complete 2025-08-09 21:50:18 -03:00
pedrocarlo
5924274d61 adjust state machine transition 2025-08-09 21:50:18 -03:00
pedrocarlo
6fe19e4ef4 adjust external sort init_chunk_heap and next_from_chunk_heap 2025-08-09 21:50:18 -03:00
pedrocarlo
c02936eb30 state machine for insert 2025-08-09 21:50:18 -03:00
pedrocarlo
2ec58b0264 state machine for sort 2025-08-09 21:50:18 -03:00
pedrocarlo
c91c22a6a8 state machine for next 2025-08-09 21:50:18 -03:00
pedrocarlo
2ffc5ee423 SorterChunk read and write should return completions 2025-08-09 21:50:18 -03:00
PThorpe92
5061617363 Fix bug in checkpoitn that sets nbackfills and lets readers skip mapping the wal 2025-08-09 16:43:05 -04:00
PThorpe92
0ba52a489e Fix check in CheckpointState::Start for work after determining safe frame 2025-08-09 15:45:47 -04:00
PThorpe92
23f30a481a Add tests for Full checkpoint mode 2025-08-09 15:45:47 -04:00
PThorpe92
cd7fb2f329 Add Full checkpoint mode in WAL 2025-08-09 15:45:45 -04:00
PThorpe92
01f281bae5 Fix doc test/comment 2025-08-09 14:41:15 -04:00
PThorpe92
3d518baef5 Fix assertion to make sure a reader or writer exists when unlock is called 2025-08-09 14:40:03 -04:00
PThorpe92
f81bc3236a Use a single packed u64 for LimboRwLock and relax memory ordering 2025-08-09 14:08:25 -04:00
PThorpe92
718114f5da Keep free api consistent with try_alloc, size instead of pages 2025-08-08 11:50:16 -04:00
PThorpe92
213d589dd1 Apply review suggestions, remove FreeEntry 2025-08-08 11:07:29 -04:00
PThorpe92
84ffed709a Round up allocation for wal frame arena to next page multiple of 64 2025-08-08 10:55:29 -04:00