Commit Graph

8649 Commits

Author SHA1 Message Date
Pekka Enberg
4efa0a57fd Merge 'translate: return parse error for unsupported join types' from Jussi Saurio
We were silently ignoring these before

Closes #3006
2025-09-10 17:14:33 +03:00
Pekka Enberg
5adb586282 s/2024/2025/ 2025-09-10 16:39:11 +03:00
Pekka Enberg
bb3fbb7962 Merge 'check freelist count in integrity check' from Jussi Saurio
Closes #3003
2025-09-10 16:15:39 +03:00
Jussi Saurio
d7ce781a2a Merge 'Enable the use of indexes in DELETE statements' from Jussi Saurio
Closes #1714
This PR enables the use of an index as the iteration cursor for a point
or range deletion operation. Main changes:
- Use `Delete` opcode for the index that is iterating the rows - avoids
unnecessary seeking on that index, since it's already positioned
correctly
- Fix delete balancing; details below:
### current state
- a deletion may cause a btree rebalancing operation
- to get the cursor back to the right place after a rebalancing, we must
remember what the deleted key was and seek to it
- right now we are using `SeekOp::LT` to move to one slot BEFORE the
deleted key, so that if we delete rows in a loop, the following `Next()`
call will put us back into the right place
### problem
- When we delete multiple rows, we always iterate forwards. Using
`SeekOp::LT` implies backwards iteration, but it works OK for table
btrees since the cursor never remains on an internal node, because table
internal cells do not have payloads. However: this behavior is
problematic for indexes because we can effectively end up skipping
visiting a page entirely. Honestly: despite spending some debugging the
_old_ code, I still don't remember what exactly causes this to happen.
:) It's one of the `iter_dir` specific behaviors in `indexbtree_move_to`
or `get_prev_record()`, but I'm too tired to spend more time figuring it
out. I had the reason in my head before going on vacation, but it was
evicted from the cache it seems...
### solution
use `SeekOp::GE { eq_only: true }` instead and make the next call to
`Next()` a no-op instead. This has the same effect as SeekOp::LT +
next(), but without introducing bugs due to `LT` being implied backwards
iteration.

Reviewed-by: Nikita Sivukhin (@sivukhin)

Closes #2981
2025-09-10 16:00:54 +03:00
Jussi Saurio
eb2710438c translate: return parse error for unsupported join types 2025-09-10 15:46:52 +03:00
Jussi Saurio
752627a213 Merge 'Add scripts that help debug bugs from simulator' from Jussi Saurio
1. Add script that cleans simulator logs into just the SQL statements
2. Add script that bisects a set of SQL statements to find the minimal
prefix set of statements that fails SQLite integrity check

Closes #2999
2025-09-10 15:44:54 +03:00
Jussi Saurio
e3594d0ae0 make the comment for skip_advance more accurate 2025-09-10 15:38:57 +03:00
Jussi Saurio
32c4f5ce81 Assert that skip_advance is not set in the middle of a seek 2025-09-10 15:38:57 +03:00
Jussi Saurio
4eb61a9527 Add gaps to update/delete fuzz test where clauses 2025-09-10 15:38:57 +03:00
Pekka Enberg
860627942a Merge 'core: Rename IO::run_once() to IO::step()' from Pekka Enberg
The `run_once()` name is just a historical accident. Furthermore, it now
started to appear elsewhere as well, so let's just call it IO::step() as
we should have from the beginning.

Closes #3001
2025-09-10 15:11:28 +03:00
Jussi Saurio
8b7c0334d4 RUFFFffff 2025-09-10 15:00:08 +03:00
Pekka Enberg
e0a3d2741b Merge 'simulator: Clean up code to use extract_if()' from Pavan Nambi
As we are in rust 1.88 rn, we can safely use extract_if

Reviewed-by: Pedro Muniz (@pedrocarlo)

Closes #2911
2025-09-10 14:59:42 +03:00
Jussi Saurio
2ff5e15f58 Add scripts that help debug bugs from simulator
1. Add script that cleans simulator logs into just the SQL statements
2. Add script that bisects a set of SQL statements to find the minimal
   prefix set of statements that fails SQLite integrity check
2025-09-10 14:56:58 +03:00
Jussi Saurio
618f51330a advance despite skip_advance flag if cursor not pointing at record 2025-09-10 14:54:51 +03:00
Jussi Saurio
80f8794fda add comments 2025-09-10 14:54:51 +03:00
Jussi Saurio
53eaf56a63 let's apply clippy's suggestion that makes the code less readable 2025-09-10 14:54:51 +03:00
Jussi Saurio
36ec654631 Seek with GE after delete balancing and skip next advance 2025-09-10 14:54:51 +03:00
Jussi Saurio
813bdc088b Adjust fuzz test to do a WHERE-less update or delete sometimes 2025-09-10 14:54:51 +03:00
Jussi Saurio
f469113d9f Don't crash if DELETE uses index 2025-09-10 14:54:51 +03:00
Jussi Saurio
e0ca0cf8af Enable access path optimizer for DELETE 2025-09-10 14:54:51 +03:00
Jussi Saurio
6d43bdbf71 emit the Delete instruction for the iteration index cursor, and do it last 2025-09-10 14:54:51 +03:00
Jussi Saurio
df83b56083 check freelist count in integrity check 2025-09-10 14:53:28 +03:00
Pekka Enberg
c620a15a55 Merge 'core: Support ceiling modifier in datetime' from Ceferino Patino
Resolves #2677
- Implements the modifier for Floor on the datetime object.
- Implements the modifier for Ceiling on the datetime object.
- Includes additional testing changes in testing/scalar-functions-
datetime.test to test the sql functionality.
Consolidation PR of #2678 and #2679 since the functions ended up being
much more complicated than I initially thought and had to be done in a
single PR.

Closes #2757
2025-09-10 14:46:07 +03:00
Pekka Enberg
a3d23c2738 Merge 'Fix jsonb functions check valid json string binary' from
closes: #2820
```
turso> select user->>'age' as age from json_user;
┌─────┐
│ age │
├─────┤
│  30 │
└─────┘
```

Closes #2821
2025-09-10 14:44:54 +03:00
Pekka Enberg
2131a04b7d core: Rename IO::run_once() to IO::step()
The `run_once()` name is just a historical accident. Furthermore, it now
started to appear elsewhere as well, so let's just call it IO::step() as we
should have from the beginning.
2025-09-10 14:36:02 +03:00
Pekka Enberg
0b91f8a715 Merge 'IO: handle errors properly in io_uring' from Preston Thorpe
Because `io_uring` may have many other I/O submission events queued
(that are relevant to the operation) when we experience an error,
marking our `Completion` objects as aborted is not sufficient, the
kernel will still execute queued I/O, which can mutate WAL or DB state
after we’ve declared failure and keep references (iovec arrays, buffers)
alive and stall reuse. We need to stop those in-flight SQEs at the
kernel and then drain the ring to a known-empty state before reusing any
resources.
The following methods were added to the `IO` trait:
`cancel`: which takes a slice of `Completion` objects and has a default
implementation that simply marks them as `aborted`.
`drain`: which has a default noop implementation, but the `io_uring`
backend implements this method to drain the ring.
CC @sivukhin

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #2787
2025-09-10 14:24:43 +03:00
Pekka Enberg
f3ee355fa1 testing/compare.test: Clean up tests by using in-memory database
Suggested by Jussi
2025-09-10 14:14:46 +03:00
Pekka Enberg
22b352bc2f Merge 'core: Fix integer/float comparison' from Pavan Nambi
closes #2957

Closes #2958
2025-09-10 14:13:15 +03:00
Pekka Enberg
1ec6233fe1 Merge 'pager: fix incorrect freelist page count bookkeeping' from Jussi Saurio
When we use a freelist trunk page, the freelist count should decrease,
not increase
file under: lmao
Closes #2992

Closes #3000
2025-09-10 14:10:48 +03:00
Pekka Enberg
ab871b70b1 Merge 'core/vdbe: Don't rollback transaction when write upgrade fails' from Pekka Enberg
If upgrade from read to write transaction fails, don't roll back the
transaction. Instead restore the transaction into its original state,
which allows deferred transactions that have not read anything to
restart automatically.
Fixes #2984

Closes #2996
2025-09-10 14:09:29 +03:00
Jussi Saurio
5c8afc5caf pager: fix incorrect freelist page count bookkeeping 2025-09-10 14:02:17 +03:00
Pekka Enberg
9655b455ed Clippy you are so smart 2025-09-10 13:54:42 +03:00
Pekka Enberg
a2f0725a62 Fix publish-create.sh script 2025-09-10 13:49:55 +03:00
Pekka Enberg
e04938eaf5 Turso 0.1.5 2025-09-10 13:30:18 +03:00
Pekka Enberg
f3d1c8c1a4 Update CHANGELOG 2025-09-10 13:30:04 +03:00
Pekka Enberg
0287512a47 stress: Disable database reopen and reconnect
Nikita noticed that database reopen and reconnet cause most database
operations to return "database locked", which means we're not getting
the coverage we need.
2025-09-10 11:50:22 +03:00
Pekka Enberg
1d34122414 Turso 0.1.5-pre.5 2025-09-10 11:40:21 +03:00
Pekka Enberg
086dac3a8c Merge 'add missing module type for browser package' from Nikita Sivukhin
Follow up after #2799

Closes #2994
2025-09-10 11:37:01 +03:00
Jussi Saurio
a874530db8 Fix semantics of transaction isolation test
- Busy errors do not rollback the transaction
- Transaction takes snapshot of DB state after its first successful
  access of the DB, not before its first attempt to access the DB
2025-09-10 11:13:50 +03:00
Jussi Saurio
11339fc941 Merge 'Fix clear_page_cache method and rollback' from Preston Thorpe
Previously we were iterating over every entry in the page cache,
clearing the dirty flag from each page.

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Reviewed-by: Nikita Sivukhin (@sivukhin)

Closes #2988
2025-09-10 11:11:37 +03:00
Jussi Saurio
ab9dbba17c Merge 'Fix read_entire_wal_dumb: incrementally build the frame cache' from Preston Thorpe
closes #2240
Incrementally build the frame cache by reading the WAL file in chunks
instead of reading the entire file into memory.
<img width="247" height="254" alt="image" src="https://github.com/user-
attachments/assets/803645ab-002a-4efd-ac47-b2f690e63fc7" />

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Reviewed-by: Nikita Sivukhin (@sivukhin)

Closes #2986
2025-09-10 11:09:42 +03:00
Jussi Saurio
b6d99cd188 Merge 'clean print_query_result' from Lâm Hoàng Phúc
different between output modes is how we handle `Ok(StepResult::Row)` so
let introduce `row_step_result_query` macro to reduce duplicated code.

Closes #2979
2025-09-10 11:08:40 +03:00
Pekka Enberg
8161badbf4 core/vdbe: Don't rollback transaction when write upgrade fails
If upgrade from read to write transaction fails, don't roll back the
transaction. Instead restore the transaction into its original state,
which allows deferred transactions that have not read anything to
restart automatically.

Fixes #2984
2025-09-10 10:51:52 +03:00
Pekka Enberg
cd88681261 Merge 'update update-script to properly handle JS workspace' from Nikita Sivukhin
Closes #2989
2025-09-10 09:18:49 +03:00
TcMits
e8b853ed25 pretty mode's table need a line break 2025-09-10 12:03:59 +07:00
Preston Thorpe
9d21f3be15 Merge 'Make 'faultless' sim profile create some indexes too' from Jussi Saurio
Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2991
2025-09-09 21:42:58 -04:00
Nikita Sivukhin
f3f69598c5 add missing module type for browser package 2025-09-10 03:06:34 +04:00
Preston Thorpe
b2638968fa Merge 'no need QueryStatistics if self.opts.timer is not set' from Lâm Hoàng Phúc
Reviewed-by: Preston Thorpe <preston@turso.tech>

Closes #2980
2025-09-09 17:59:32 -04:00
Jussi Saurio
f5a15bfb3d run 50% of AWS sims with faultless profile 2025-09-10 00:38:37 +03:00
Jussi Saurio
23c55eab6c Make 'faultless' sim profile create some indexes too 2025-09-10 00:32:09 +03:00