Commit Graph

60 Commits

Author SHA1 Message Date
pedrocarlo
e93add6c80 remove dyn DatabaseStorage and replace it with DatabaseFile 2025-10-03 14:14:15 -03:00
Duy Dang
8562737d77 sum() can throw integer overflow 2025-09-29 00:27:59 +07:00
Pavan-Nambi
020921f803 Merge remote-tracking branch 'upstream/main' into cdc_fail_autoincrement 2025-09-18 19:27:19 +05:30
rajajisai
89caa868f9 Encryption support for database header page 2025-09-16 10:04:30 -04:00
Pavan-Nambi
cfe01eafaa fmt 2025-09-12 16:53:12 +05:30
Pavan-Nambi
8da4e30308 update rootpage and rowid in test_cdc 2025-09-12 16:21:23 +05:30
Pekka Enberg
2131a04b7d core: Rename IO::run_once() to IO::step()
The `run_once()` name is just a historical accident. Furthermore, it now
started to appear elsewhere as well, so let's just call it IO::step() as we
should have from the beginning.
2025-09-10 14:36:02 +03:00
Nikita Sivukhin
47808e9da8 enable tracing subscriber in integration tests 2025-09-07 19:56:06 +04:00
Pekka Enberg
12cf4d2e72 core: Make strict schema support experimental
It's not tested properly so let's mark it as experimental for now.

Fixes #2775
2025-09-02 16:40:02 +03:00
Nikita Sivukhin
c62b87d9b6 read from database file only if max_frame_read_lock_index is 0 and max_frame > min_frame
- transaction which was started with max_frame = 0 and max_frame_read_lock_index = 0
  can write to the WAL and in this case it needs to read data back from WAL
- without cache spilling its hard to reproduce this issue for the turso-db now,
  but I stumbled into this issue with sync-engine which do weird stuff with the WAL
  which "simulates" cache spilling behaviour to some extent
2025-08-25 11:36:58 +04:00
Nikita Sivukhin
fdb80746be fix clippy 2025-08-21 18:28:16 +04:00
Nikita Sivukhin
b7c033be20 add tests 2025-08-21 18:28:16 +04:00
Nikita Sivukhin
f6729ccc36 fix clippy 2025-08-21 18:27:54 +04:00
Nikita Sivukhin
89e180c2a8 add few tests 2025-08-21 18:27:54 +04:00
Nikita Sivukhin
d7e47c1268 fix bug - continue checkpoint as usual even if frames range is degenerate 2025-08-21 17:37:19 +04:00
Nikita Sivukhin
c34d884b6e fix tests 2025-08-21 15:24:21 +04:00
Nikita Sivukhin
144a31192f fix clippy 2025-08-21 14:13:26 +04:00
Nikita Sivukhin
05931f70ce add optional upper_bound_inclusive parameter to some checkpoint modes
- will be used in sync-engine protocol
2025-08-21 14:12:11 +04:00
Nikita Sivukhin
5d0ada9fb9 add "updates" column for cdc table 2025-08-11 12:46:15 +04:00
Nikita Sivukhin
c0d5c55d5c fix tests and clippy 2025-08-06 01:03:49 +04:00
Nikita Sivukhin
9c4147e8d6 add simple integration tests 2025-08-06 01:03:49 +04:00
Nikita Sivukhin
76bdf0c1ab small fixes 2025-08-04 17:02:53 +04:00
Nikita Sivukhin
2e23230e79 extend raw WAL API with few more methods
- try_wal_watermark_read_page - try to read page from the DB with given WAL watermark value
- wal_changed_pages_after - return set of unique pages changed after watermark WAL position
2025-08-04 16:55:50 +04:00
Nikita Sivukhin
83b1e99a61 fix compilation 2025-08-04 12:53:07 +04:00
Jussi Saurio
438cbf2872 test/wal api: add comment about purpose of rollback test 2025-07-30 18:17:07 +03:00
Nikita Sivukhin
d8be1cbef1 fix after rebase 2025-07-28 17:20:57 +04:00
Nikita Sivukhin
4d25cda1e2 slightly adjust one test 2025-07-28 17:20:10 +04:00
Nikita Sivukhin
eb32ea49e6 fix tests 2025-07-28 17:20:10 +04:00
Nikita Sivukhin
27fcb81f48 add more complex schema changes test for raw WAL API 2025-07-24 22:43:31 +04:00
Nikita Sivukhin
10836510df remove tracing_subscriber 2025-07-24 11:52:07 +04:00
Nikita Sivukhin
6daa6d07f1 re-parse schema if necessary after WAL sync end 2025-07-24 11:52:07 +04:00
Nikita Sivukhin
fb83862013 fix clippy 2025-07-24 11:49:39 +04:00
Nikita Sivukhin
435ca7fe7a add fuzz tests for raw WAL API 2025-07-24 11:49:39 +04:00
Nikita Sivukhin
2283a04aab add more tests 2025-07-23 11:31:00 +04:00
Nikita Sivukhin
16763e1500 implement raw WAL write api 2025-07-23 11:30:59 +04:00
Jussi Saurio
022f679fab chore: make every CREATE TABLE stmt in entire repo have 1 space after tbl name
`BTreeTable::to_sql` makes us incompatible with SQLite by losing e.g. the original whitespace provided during the CREATE TABLE command.

For now let's fix our tests by regex-replacing every CREATE TABLE in
the entire repo to have exactly 1 space after the table name in the
CREATE TABLE statement.
2025-07-22 11:35:21 +03:00
Nils Koch
8dc066503e chore: fix clippy errors 2025-07-16 19:34:42 +01:00
Nikita Sivukhin
97b82fe6d8 rename operation_xxx to change_xxx to make naming more consistent 2025-07-16 20:16:24 +04:00
Nikita Sivukhin
551c353fff fix clippy 2025-07-14 11:27:51 +04:00
Nikita Sivukhin
81cd04dd65 add bin_record_json_object and table_columns_json_array functions 2025-07-14 11:19:45 +04:00
Nikita Sivukhin
5409812610 properly implement generation of before/after records for new modes 2025-07-14 11:17:32 +04:00
Nikita Sivukhin
9e04102a94 add basic cdc tests for new modes 2025-07-14 11:17:31 +04:00
Nikita Sivukhin
fabb00f385 fix test 2025-07-14 11:16:06 +04:00
Nils Koch
828d4f5016 fix clippy errors for rust 1.88.0 (auto fix) 2025-07-12 18:58:41 +03:00
Pekka Enberg
341f963a8e Merge 'Fix infinite loops, rollback problems, and other bugs found by I/O fault injection' from Pedro Muniz
Was running the sim with I/O faults enabled and fixed some nasty bugs.
Now, there are some more nasty bugs to fix as well. This is the command
that I use to run the simulator `cargo run -p limbo_sim -- --minimum-
tests 10 --maximum-tests 1000`
This PR mainly fixes the following bugs:
- Not decrementing in flight write counter when `pwrite` fails
- not rolling back the transaction on `step` error
- not rolling back the transaction on `run_once` error
- some functions were just being unwrapped when they could suffer io
errors
- Only change max_frame after wal sync's

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1946
2025-07-07 21:31:26 +03:00
pedrocarlo
711b1ef114 make all run_once be run under statement or connection so that rollback is called 2025-07-07 11:51:25 -03:00
Nikita Sivukhin
1655c0b84f small fixes 2025-07-07 12:50:10 +04:00
Nikita Sivukhin
3e7e66c0e7 add basic cdc tests 2025-07-07 12:44:50 +04:00
Pekka Enberg
53ba3ff926 Rename limbo_core crate to turso_core 2025-06-29 09:59:17 +03:00
Pekka Enberg
2fc5c0ce5c Switch to runtime flag for enabling indexes
Makes it easier to test the feature:

```
$ cargo run --  --experimental-indexes
Limbo v0.0.22
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database
limbo> CREATE TABLE t(x);
limbo> CREATE INDEX t_idx ON t(x);
limbo> DROP INDEX t_idx;
```
2025-06-26 10:07:28 +03:00