Commit Graph

5437 Commits

Author SHA1 Message Date
Jussi Saurio
ba3bfa058d Delete unused stuff 2025-06-24 14:41:50 -03:00
Jussi Saurio
6091d5abd7 Add todo comment 2025-06-24 14:41:50 -03:00
Jussi Saurio
fcc070c27e Remove unused method 2025-06-24 14:41:50 -03:00
Jussi Saurio
8d4ed110c0 alloc page1 on first tx (read OR write) - otherwise e.g. select * from sqlite_schema panics 2025-06-24 14:41:50 -03:00
Jussi Saurio
d05030ed64 fix autovacuum test 2025-06-24 14:41:50 -03:00
Jussi Saurio
4b5151d2c6 Fix not setting Database::maybe_shared_wal to Some after creating WAL 2025-06-24 14:41:50 -03:00
Diego Reis
43aa2577cd Implement size for JS and WASM storages 2025-06-24 14:41:50 -03:00
Diego Reis
af193ef76a Move first page allocation logic to begin_write_tx 2025-06-24 14:41:50 -03:00
Jussi Saurio
62e52e0148 only initialize pager in empty state when no WAL and db file size 0 2025-06-24 14:41:50 -03:00
Jussi Saurio
acafe71d9e Dont allocate fresh page1 if it's in the WAL 2025-06-24 14:41:50 -03:00
Jussi Saurio
dc13d88900 Read schema from nonempty WAL if present 2025-06-24 14:41:50 -03:00
Jussi Saurio
9aae1ff00a sim: add Fault::ReopenDatabase
Add fault type that reopens the DB and reconnects the connections.
It purposefully does not call conn.close(), as the default behavior
of that method is to checkpoint the database.

This easily exposes multiple manifestations of DB/WAL corruption caused by
this issue: https://github.com/tursodatabase/limbo/issues/1725

in fact, in my testing, every run fails when this fault is enabled. Hence I've
added the --disable-reopen-database flag if you want to try to find some other
bugs.
2025-06-24 14:41:50 -03:00
Diego Reis
f0f9ad4844 core: Get rid of maybe_init_database_file
Initialization now only occurs in the first write transaction
2025-06-24 14:41:50 -03:00
Diego Reis
ce73025202 core: Only actually allocate page1 on first write transaction 2025-06-24 14:41:50 -03:00
Diego Reis
2f33c799e3 core: Set default database_size to 0 2025-06-24 14:41:49 -03:00
Diego Reis
75fdbd73c6 core: Add count of pages in Pager and fix page1 initialization 2025-06-24 14:41:49 -03:00
Diego Reis
9c7330c01c core: Add size method to DatabaseStorage trait 2025-06-24 14:41:49 -03:00
Jussi Saurio
cc2e14b11c Read page 1 from pager always, no separate db_header 2025-06-24 14:41:49 -03:00
Pekka Enberg
f01516378b Merge 'Add Pedro to email recipients for antithesis' from Pedro Muniz
I want to be notified on Antithesis test stuff. I want to understand
more how their stuff works and what fails and how in their platform.

Closes #1785
2025-06-24 16:12:31 +03:00
Pekka Enberg
2313c33015 Merge 'bindings/rust: Implement Debug for Connection' from Charlie
A simple change to implement the `Debug` trait for the `Connection`,
similar to how it is implemented for `Database`.  This should help users
in their application code with wrapping the connection.

Closes #1798
2025-06-24 16:11:54 +03:00
Pekka Enberg
b1cd7229c7 Merge 'sim: when loading bug, dont panic if there are no runs' from Jussi Saurio
Closes #1820
2025-06-24 16:07:52 +03:00
Pekka Enberg
69d8162e61 Merge 'sim: fix singlequote escaping and unescaping' from Jussi Saurio
Fixes https://github.com/tursodatabase/limbo/issues/1801 by preventing
the generation of strings like `'foo''` that contain an improper
unescaped sequence of quotes

Closes #1819
2025-06-24 16:07:43 +03:00
Pekka Enberg
28c2d24931 antithesis: Relax connect failure check
We also want to just ignore this under Antithesis:

``´
limbo.DatabaseError: Failed to open database: LockingError("Failed locking file. File is locked by another process")
```
2025-06-24 15:47:21 +03:00
Jussi Saurio
73db450b8b sim: when loading bug, dont panic if there are no runs 2025-06-24 15:46:15 +03:00
Jussi Saurio
cff96964df Fix singlequote escaping and unescaping 2025-06-24 14:58:25 +03:00
Pekka Enberg
6c8aa4e043 antithesis: Run experiment for 60 minutes 2025-06-24 14:16:09 +03:00
Pekka Enberg
f8c2c7387f antithesis: Reduce number of stress iterations
...currently takes a very long time to run.
2025-06-24 14:15:46 +03:00
Jussi Saurio
0a94ee5572 Merge 'Fix btree balance and seek after overwritten cell overflows' from Jussi Saurio
Closes #1809
Closes #1810
Closes #1811
Closes #1812
Closes #1813
Closes #1814
Closes #1815
and probably Closes #1805

Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1816
2025-06-24 13:29:53 +03:00
Jussi Saurio
27b2c2530b test/stress: only do integrity_check every 100 queries 2025-06-24 13:13:31 +03:00
Jussi Saurio
bdfbb8fe54 Fix erroneous early return 2025-06-24 11:26:00 +03:00
Jussi Saurio
5878724d0e fix/btree: balance and seek after overwritten cell overflows 2025-06-24 11:08:22 +03:00
Pekka Enberg
a99833ba8d Merge 'Bring back some merge conflicts code' from Pedro Muniz
Due to some merge conflicts, some recent code from stress testing was
lost. Bringing it back here

Closes #1808
2025-06-24 09:07:32 +03:00
pedrocarlo
13a55948d5 bring back some merge conflicts code 2025-06-23 23:01:28 -03:00
Jussi Saurio
e2d8d11738 Merge 'chore: fix clippy warnings' from Nils Koch
There are a many clippy warnings in the main branch. This PR addresses
some of them.
I will fix the rest of the warnings in follow up PRs. Furthermore, the
clippy CI check is not working. I can look into that after I fixed the
rest of the clippy warnings. After that, I think it would make sense to
migrate to the latest Rust version. Does that work for you?
EDIT: This PR resolves all Clippy warnings and enables the CI to run all
Clippy checks. The CI has extra checks for the `limbo-wasm` module. This
module still has some Clippy warnings that need attention.

Reviewed-by: Jussi Saurio <jussi.saurio@gmail.com>
Reviewed-by: Pere Diaz Bou <pere-altea@homail.com>

Closes #1742
2025-06-23 23:22:21 +03:00
Nils Koch
4c7fcafe77 ci: fail on all clippy warnings 2025-06-23 19:52:13 +01:00
Nils Koch
2827b86917 chore: fix clippy warnings 2025-06-23 19:52:13 +01:00
Pekka Enberg
1a0d4f500f Merge 'simulator: integrity check per query' from Pedro Muniz
Now that we have our own integrity check, we can also execute it per
query to catch bugs earlier. We could use `Sqlite`s integrity check
instead of the one @pereman2 implemented, but I think this could also
help us battle test the integrity check he wrote.
Motivation for this PR came from this issue #1781 . I wanted to find a
minimal repo for that problem, so I introduced this check per query. The
nice thing is that we get some shrinking as well by using the simulator
instead of `limbo_stress`.
The following command will allow you to find the failing seed I posted
in the issue:
`cargo run -p limbo_sim -- --minimum-tests 10 --maximum-tests 100
--disable-create-index --seed 17321136847878547364`

Closes #1802
2025-06-23 20:17:32 +03:00
Pekka Enberg
8686d9bb49 Merge 'don't emit Delete for UPDATE statement' from Pere Diaz Bou
Previously we implemented update as a simple `Delete` + `Insert`
procedure which seemed okay for the moment but it wasn't. `Delete` can
trigger balance and a post balance `seek` which will leave cursor
pointing to an invalid page which `Insert` will try to insert to.
We solve this by removing `Delete` from the execution plan and rely on
`Insert` to properly overwrite the cell where the rowid is the same as
the one we are inserting.

Closes #1803
2025-06-23 20:16:16 +03:00
Pere Diaz Bou
404b5fad8a clippy 2025-06-23 18:19:44 +02:00
Pere Diaz Bou
b7fd4f1e18 update nchanges on op_insert 2025-06-23 18:11:58 +02:00
Pekka Enberg
43d8799f55 Merge 'stress: Improve progress reporting' from Pekka Enberg
A stress run can take a long time. Improve progress reporting to improve
DX.

Closes #1796
2025-06-23 17:48:55 +03:00
Pekka Enberg
c0fd559f92 Merge 'Fix: add uv sync to all packages for pytest github action' from Pedro Muniz
Hopefully, this solves the problem in main branch.

Closes #1800
2025-06-23 17:48:44 +03:00
Pekka Enberg
963d7b7daf stress: Improve progress reporting
A stress run can take a long time. Improve progress reporting to improve
DX.
2025-06-23 17:25:38 +03:00
Pere Diaz Bou
c7e9b3a546 don't emit Delete for UPDATE statement
Previously we implemented update as a simple `Delete` + `Insert`
procedure which seemed okay for the moment but it wasn't. `Delete` can
trigger balance and a post balance `seek` which will leave cursor
pointing to an invalid page which `Insert` will try to insert to.

We solve this by removing `Delete` from the execution plan and rely on
`Insert` to properly overwrite the cell where the rowid is the same as
the one we are inserting.
2025-06-23 15:44:34 +02:00
pedrocarlo
c9b11dd173 simulator integrity check per query 2025-06-22 21:47:17 -03:00
pedrocarlo
efc1516d7b add uv sync to all packages 2025-06-22 15:45:07 -03:00
CM-IV
3bb0374401 impl Debug for Connection 2025-06-22 11:41:07 -05:00
Pekka Enberg
b07e95b892 Merge 'Fix CI errors' from Piotr Rżysko
Fix CI errors caused by https://github.com/tursodatabase/limbo/pull/1741

Closes #1794
2025-06-22 09:24:12 +03:00
Piotr Rzysko
ddd5417978 Add sqlite_test_ext to Dockerfile.antithesis 2025-06-21 20:07:26 +02:00
Piotr Rzysko
0f3efe2fbb Copy sqlite_test_ext to Dockerfile.simulator 2025-06-21 19:51:11 +02:00