Commit Graph

171 Commits

Author SHA1 Message Date
Jussi Saurio
347a9152a6 Merge 'Replace verbose IO Completion methods with helpers' from Preston Thorpe
one of the last remnants of some original verbosity

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2156
2025-07-18 10:52:17 +03:00
Jussi Saurio
bbd7f32d80 Merge 'Fix rollback for TxErrors' from Diego Reis
Fixes #2153.
Not so sure if SQLite doesn't rollback in more cases, we should
definitively check this out.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2154
2025-07-18 10:49:29 +03:00
Diego Reis
9ade79b3ad Do not rollback in TxError
Fixes #2153. Not so sure if SQLite doesn't rollback in more cases, we
should definitively check this out.
2025-07-18 01:11:51 -03:00
PThorpe92
a0a948101d Last cleanups of old completion api 2025-07-17 23:47:43 -04:00
Jussi Saurio
1b52b5c764 Merge 'chore: update rust to version 1.88.0' from Nils Koch
This PR updates to version Rust 1.88.0 ([Release
notes](https://releases.rs/docs/1.88.0/)) and fixes all the clippy
errors that come with the new Rust version.
This is possible in the latest Rust version:
```rust
if let Some(foo) = bar && foo.is_cool() {
  ...
}
```
There are three complications in the migration (so far):
- A BUNCH of Clippy warnings (mostly fixed in
https://github.com/tursodatabase/limbo/pull/1827)
- Windows cross compilation failed; linking `advapi32` on windows fixes
it
  - Since Rust 1.87.0, advapi32 is not linked by default anymore
([Release notes](https://github.com/rust-
lang/rust/blob/master/RELEASES.md#compatibility-notes-1),
[PR](https://github.com/rust-lang/rust/pull/138233))
- Rust is more strict with FFIs and aligning pointers now. CI checks
failed with error below
  - Fixed in https://github.com/tursodatabase/turso/pull/2064
```
thread 'main' panicked at
core/ext/vtab_xconnect.rs:64:25:
misaligned pointer dereference: address must be
a multiple of 0x8 but is 0x7ffd9d901554
```

Closes #1807
2025-07-17 23:35:33 +03:00
pedrocarlo
b80218324d fix merge conflicts 2025-07-17 12:25:31 -03:00
Nils Koch
8dc066503e chore: fix clippy errors 2025-07-16 19:34:42 +01:00
Pekka Enberg
e6c3a5a9b8 Merge 'rename operation_xxx to change_xxx to make naming more consistent' from Nikita Sivukhin
This PR renames CDC table column names to use "change"-centric
terminology and avoid using `operation_xxx` column names.
Just a small refactoring to bring more consistency as `turso-db` refer
to the feature as capture data **changes** - and there is no word
operation here.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>

Closes #2120
2025-07-16 20:08:23 +03:00
Nikita Sivukhin
97b82fe6d8 rename operation_xxx to change_xxx to make naming more consistent 2025-07-16 20:16:24 +04:00
Diego Reis
817ad8d50f Separate user-callable cacheflush from internal cacheflush logic
Cacheflush should only spill pages to WAL as non-commit frames, without checkpointing nor syncing. Check SQLite's sqlite3PagerFlush
2025-07-16 11:08:50 -03:00
Diego Reis
0e9771ac07 refactor: Change redundant "Status" enums to IOResult
Let's unify the semantics of "something done" or yields I/O into a
single type
2025-07-15 20:56:18 -03:00
Diego Reis
d0af54ae77 refactor: Change CursorResult to IOResult
The reasoning here is to treat I/O operations (Either is "Done" or
yields to IO) with the same generic type.
2025-07-15 20:52:25 -03:00
Jussi Saurio
615ccf6789 test/fuzz: fix rowid_seek_fuzz
The original `rowid_seek_fuzz` test had a design flaw: it inserted contiguous
integers, which prevented issues such as the one fixed in #2065 from being
discovered.

Further, the test has at some point also been neutered a bit by only inserting
100 values which makes the btree very small, hiding interactions between interior
pages and neighboring leaf pages.

This should not be merged until #2065 is merged.
2025-07-14 12:57:59 +03:00
Nikita Sivukhin
82773d6563 fix clippy 2025-07-14 13:01:15 +04:00
Nikita Sivukhin
47ab260f6c use PlatformIO in the fuzz test code 2025-07-14 13:01:15 +04:00
Nikita Sivukhin
c4841e18f3 fix clippy 2025-07-14 13:01:15 +04:00
Nikita Sivukhin
6e2ccdff20 add btree fuzz tests which generate seed file from scratch 2025-07-14 13:01:15 +04:00
Nikita Sivukhin
551c353fff fix clippy 2025-07-14 11:27:51 +04:00
Nikita Sivukhin
81cd04dd65 add bin_record_json_object and table_columns_json_array functions 2025-07-14 11:19:45 +04:00
Nikita Sivukhin
5409812610 properly implement generation of before/after records for new modes 2025-07-14 11:17:32 +04:00
Nikita Sivukhin
9e04102a94 add basic cdc tests for new modes 2025-07-14 11:17:31 +04:00
Nikita Sivukhin
fabb00f385 fix test 2025-07-14 11:16:06 +04:00
Jussi Saurio
a48b6d049a Another post-rebase clippy round with 1.88.0 2025-07-12 19:10:56 +03:00
Nils Koch
1a91966c7e fix clippy errors for rust 1.88.0 (manual fix) 2025-07-12 18:58:55 +03:00
Nils Koch
828d4f5016 fix clippy errors for rust 1.88.0 (auto fix) 2025-07-12 18:58:41 +03:00
Jussi Saurio
897f59fab1 test/fuzz: add ignored fuzz test for min()/max() - ignored because of bugs 2025-07-10 21:02:57 +03:00
Pere Diaz Bou
832f9fb8a8 clippy 2025-07-10 12:23:33 +02:00
Pere Diaz Bou
9258d33d8b properly set last_checksum after recovering wal
We store `last_checksum` to do cumulative checksumming. After reading
wal for recovery, we didn't set last checksum properly in case there
were no frames so this cause us to not initialize last_checksum
properly.
2025-07-10 12:15:00 +02:00
meteorgan
f44d818400 cargo fmt 2025-07-08 22:57:20 +08:00
meteorgan
6768f073c8 add tests for except operator 2025-07-08 22:57:20 +08:00
Pekka Enberg
341f963a8e Merge 'Fix infinite loops, rollback problems, and other bugs found by I/O fault injection' from Pedro Muniz
Was running the sim with I/O faults enabled and fixed some nasty bugs.
Now, there are some more nasty bugs to fix as well. This is the command
that I use to run the simulator `cargo run -p limbo_sim -- --minimum-
tests 10 --maximum-tests 1000`
This PR mainly fixes the following bugs:
- Not decrementing in flight write counter when `pwrite` fails
- not rolling back the transaction on `step` error
- not rolling back the transaction on `run_once` error
- some functions were just being unwrapped when they could suffer io
errors
- Only change max_frame after wal sync's

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1946
2025-07-07 21:31:26 +03:00
pedrocarlo
711b1ef114 make all run_once be run under statement or connection so that rollback is called 2025-07-07 11:51:25 -03:00
Nikita Sivukhin
1655c0b84f small fixes 2025-07-07 12:50:10 +04:00
Nikita Sivukhin
3e7e66c0e7 add basic cdc tests 2025-07-07 12:44:50 +04:00
Pekka Enberg
36a7f76708 tests/integration: Fix write path test on Windows
The read_at() and write_at() APIs are UNIX specific.
2025-06-30 14:38:54 +03:00
Pekka Enberg
f9967f809a tests/integration: s/limbo_core/turso_core/g 2025-06-30 14:00:55 +03:00
Pekka Enberg
51edab7032 Merge 'WAL record db_size frame on commit last frame' from Pere Diaz Bou
`db_size` is `>0` in case of last frame written of a transaction. This
is necessary as we need to know -- while recovering wal contents -- that
we have read a transaction fully instead of treating every frame as its
own transaction.

Closes #1866
2025-06-30 13:46:40 +03:00
Pere Diaz Bou
ec6e39f426 test wal mismtach checksum 2025-06-30 12:24:58 +02:00
Pekka Enberg
9c1b7897ac Fix URLs to point to github.com/tursodatabase/turso 2025-06-30 11:23:53 +03:00
Pekka Enberg
53ba3ff926 Rename limbo_core crate to turso_core 2025-06-29 09:59:17 +03:00
Pere Diaz Bou
22cb95bc5d execute run on multiple parsed statements 2025-06-27 14:02:47 +02:00
meteorgan
c060905d00 add INTERSECT to compound_select_fuzz 2025-06-27 11:50:19 +08:00
Pekka Enberg
2fc5c0ce5c Switch to runtime flag for enabling indexes
Makes it easier to test the feature:

```
$ cargo run --  --experimental-indexes
Limbo v0.0.22
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database
limbo> CREATE TABLE t(x);
limbo> CREATE INDEX t_idx ON t(x);
limbo> DROP INDEX t_idx;
```
2025-06-26 10:07:28 +03:00
Nils Koch
2827b86917 chore: fix clippy warnings 2025-06-23 19:52:13 +01:00
Pere Diaz Bou
c7e9b3a546 don't emit Delete for UPDATE statement
Previously we implemented update as a simple `Delete` + `Insert`
procedure which seemed okay for the moment but it wasn't. `Delete` can
trigger balance and a post balance `seek` which will leave cursor
pointing to an invalid page which `Insert` will try to insert to.

We solve this by removing `Delete` from the execution plan and rely on
`Insert` to properly overwrite the cell where the rowid is the same as
the one we are inserting.
2025-06-23 15:44:34 +02:00
Luca Muscat
7cf77fb35b Fix fuzz issue #1763 by using the log2 & log10 functions where applicable
It is easy to chalk this fuzzer issue to erratic floating point
behaviour, but this is not the case here.

Currently, `exec_math_log` calculates log with arbitrary bases by using
the following formula: `log_a(b) ~= ln(b) / ln(a)`. This calculation is
an approximation with lots of its floating point precision lost to
dividing the results of natural logarithms.

By using the specialized versions of the log functions (`log2` &
`log10`), we can avoid this loss of precision.

SQLite also uses these specialized log functions when possible, so it
doesn't hurt to do the same thing when aiming for parity.
2025-06-20 10:52:13 +02:00
Pere Diaz Bou
e681e32198 fix test_limbo_open_read_only 2025-06-19 10:56:01 +02:00
Pere Diaz Bou
10d02525d6 introduce concurrent write test
The idea is quite simple: write with 4 concurrent writers and once all
are finsihed, check the count of rows written is correct.
2025-06-18 17:40:53 +02:00
Pere Diaz Bou
34592b172c run index tests with flags instead of ignore 2025-06-17 19:33:23 +02:00
Pere Diaz Bou
de1f29dadf core_tester index fuzz tests with flag 2025-06-17 19:33:23 +02:00