The codebase was used to correctly perform signatory calls during a database
transaction, as the signatory was previously exclusively in process. However, a
few months ago, it was changed to be a trait that can be either local or
remote. Making external calls to services, adding latency, during an ongoing
database transaction is a bad idea because it will lock the rows until the
service call is finalized, which is unpredictable.
The issue is even worse in our pipeline where the SQLite storage driver is used
with the ":memory:" path, which forces the Database pool to have a size of 1.
Since our tests run in parallel, they would randomly fail.
This issue was failing in the CI, but the error was not making the pipeline
fail. This bug was fixed as well.
* feat: refresh keysets
* fix(cdk): resolve keyset counter skipping index 0 in deterministic secret generation
- Modified Database::get_keyset_counter to return u32 instead of Option<u32>
- Added database migrations to increment existing keyset counters by 1
- Removed counter increment logic from wallet operations to use actual counter value
- Ensures deterministic secret generation starts from index 0 instead of skipping it
The proof_writer implicit rollback on Drop is too slow for our tests causing
race conditions with Postgres.
This commit enhances the rollback logic and makes it explicit
These functions are designed as a single funnel to talk to the database,
whether it is synchronous or asynchronous.
This single funnel will log SQL queries and slow operations, providing a clear
and unified debug message for the problematic query, so it can be optimized
accordingly (for instance, missing indexes or unbound SQL requests).
* Working on a better database abstraction
After [this question in the chat](https://matrix.to/#/!oJFtttFHGfnTGrIjvD:matrix.cashu.space/$oJFtttFHGfnTGrIjvD:matrix.cashu.space/$I5ZtjJtBM0ctltThDYpoCwClZFlM6PHzf8q2Rjqmso8)
regarding a database transaction within the same function, I realized a few
design flaws in our SQL database abstraction, particularly regarding
transactions.
1. Our upper abstraction got it right, where a transaction is bound with `&mut
self`, so Rust knows how to handle its lifetime with' async/await'.
2. The raw database does not; instead, it returns &self, and beginning a
transaction takes &self as well, which is problematic for Rust, but that's not
all. It is fundamentally wrong. A transaction should take &mut self when
beginning a transaction, as that connection is bound to a transaction and
should not be returned to the pool. Currently, that responsibility lies with
the implementor. If a mistake is made, a transaction could be executed in two
or more connections.
3. The way a database is bound to our store layer is through a single struct,
which may or may not internally utilize our connection pool. This is also
another design flow, in this PR, a connection pool is owned, and to use a
connection, it should be requested, and that connection is reference with
mutable when beginning a transaction
* Improve the abstraction with fewer generics
As suggested by @thesimplekid
* Add BEGIN IMMEDIATE for SQLite
* feat: add keyset u32 mapping migration and duplicate handling
- Add new database migration (version 3) to include u32 representation for keysets
- Implement migration for both redb and SQL databases
- Add duplicate detection and handling for keyset entries
- Create unique index constraint for keyset_u32 column in SQL
- Update keyset storage to include u32 identifiers
- Handle backwards compatibility for existing databases
* chore: clippy
* refactor(cashu): simplify keyset ID verification logic
- Consolidate match expression into a single expression
- Use direct comparison with ensure_cdk macro
- Improve readability of keyset ID validation
* refactor(cdk): rename `fetch_keyset_keys` to `load_keyset_keys` for clarity
- Renamed `fetch_keyset_keys` to `load_keyset_keys` across multiple modules to better reflect its behavior of loading keys from local storage or fetching from mint when missing.
- Added debug logging to indicate when keys are being fetched from the mint.
- Simplified key loading logic in `update_mint_keysets` by removing redundant existence checks.
* chore: remove unused vec
* feat(cdk-integration-tests): refactor regtest setup and mintd integration
- Replace shell-based regtest setup with Rust binary (start_regtest_mints)
- Add cdk-mintd crate to workspace and integration tests
- Improve environment variable handling for test configurations
- Update integration tests to use proper temp directory management
- Remove deprecated start_regtest.rs binary
- Enhance CLN client connection with retry logic
- Simplify regtest shell script (itests.sh) to use new binary
- Fix tracing filters and improve error handling in setup
- Update dependencies and configurations for integration tests
fix: killing
chore: comment tests for ci debugging
chore: compile
Revert "chore: comment tests for ci debugging"
This reverts commit bfc594c11cf37caeaa6445cb854ae5567d2da6bd.
* chore: sql cipher
* fix: removal of sqlite cipher
* fix: auth password
* refactor(cdk-mintd): improve database password handling and function signatures
- Pass database password as parameter instead of parsing CLI args in setup_database
- Update function signatures for run_mintd and run_mintd_with_shutdown to accept db_password
- Remove direct CLI parsing from database setup logic
- Fix auth database initialization to use correct type when sqlcipher feature enabled
The primary purpose of this new crate is to have a common and shared codebase
for all SQL storage systems. It would force us to write standard SQL using best
practices for all databases.
This crate has been extracted from #878
Custom currency units were incorrectly being converted to uppercase
during deserialization. This change ensures custom units maintain
their original case while still allowing case-insensitive matching
for standard units.
- Add background service management with graceful shutdown handling
- Refactor invoice payment monitoring to use centralized task supervision
- Update cdk-mintd and integration tests to use new lifecycle methods
- Add comprehensive documentation and lifecycle tests
