# Implement Saga Pattern for Swap Operations with Recovery Mechanism
## Overview
This PR refactors the swap operation implementation to use the saga pattern - a distributed transaction pattern that provides reliable transaction management through explicit state tracking and compensation-based error handling. The implementation includes a robust recovery mechanism that automatically handles swap operations interrupted by crashes, power loss, or network failures.
## What Changed
**Saga Pattern Implementation:**
- Introduced a strict linear state machine for swaps: `Initial` → `SetupComplete` → `Signed` → `Completed`
- New modular `swap_saga` module with state validation, compensation logic, and saga orchestration
- Automatic rollback of database changes on failure, ensuring atomic swap operations
- Replaced previous swap implementation (`swap.rs`, `blinded_message_writer.rs`) with saga-based approach
**Recovery Mechanism:**
- Added `operation_id` and `operation_kind` columns to database schema for tracking which operation proofs belong to
- New `recover_from_bad_swaps()` method that runs on mint startup to handle incomplete swaps
- For proofs left in `PENDING` state from swap operations:
- If blind signatures exist: marks proofs as `SPENT` (swap completed but not finalized)
- If no blind signatures exist: removes proofs from database (swap failed partway through)
- Database migrations included for both PostgreSQL and SQLite
Add state machine validation for melt quote transitions to prevent
invalid state changes. Includes new error types and validation logic
for Unpaid, Pending, Paid, and Failed states.
* fix(wallet): move transaction ID calculation before database operations
* fix(sql): remove on ys from on conflict transaction insert
Since the id is created from the ys we know that if there is a conflict
the ys are the same and do not need to be updated.
* feat: bench for transactio id
* chore: fmt
Add payment_request and payment_proof fields to Transaction model to store Lightning invoice details and payment preimages. Update database migrations and all transaction creation points across wallet operations (mint, melt, send, receive) to populate these fields.
* feat: optimize SQL balance calculation
replace proof-fetching approach with SUM aggregation
- add get_balance() method to Database trait
- implement SQL SUM aggregation in cdk-sql-common
- update total_balance() to use get_balance() instead of get_unspent_proofs()
- redb impl maintains existing behavior
---------
Co-authored-by: thesimplekid <tsk@thesimplekid.com>
Co-authored-by: Cesar Rodas <cesar@rodasm.com.py>
* fix(cdk): prevent duplicate blinded message processing with database constraints
Add unique constraints on blinded_message column in both PostgreSQL and SQLite databases, and implement application-level checks to prevent duplicate blinded messages from being processed. Also ensure proper cleanup of melt requests after successful processing.
* feat: db tests for unique
* refactor(cdk-sql): consolidate blinded messages into blind signature table
Migrate from separate blinded_messages table to unified blind_signature table.
Add signed_time column and make c column nullable to track both pending
blind messages (c=NULL) and completed signatures. Update insert/update
logic to handle upsert scenarios for blind signature completion.
* refactor(cdk-sql): remove unique constraint migration and filter queries for signed messages
Remove database-level unique constraint on blinded_message and instead filter
queries to only consider messages with signatures (c IS NOT NULL
* refactor(database): improve blinded message duplicate detection using database constraints
Replace manual duplicate checking with database constraint handling for better
reliability and simplified code flow in melt request processing.
* refactor(cdk-sql): optimize blind signature processing with batch queries
Replace individual queries per blinded message with single batch query
and HashMap lookup to eliminate N+1 query performance issue.
* fix: signed time to swap sigs
* refactor(cdk): split blinded message handling and improve duplicate detection
- Split add_melt_request_and_blinded_messages into separate methods
- Add blinded messages to database before signing in swap operations
- Improve duplicate output detection with proper error handling
- Make add_blinded_messages method accept optional quote_id for flexibility
* refactor(cdk): add BlindedMessageWriter for improved transaction rollback
- Add BlindedMessageWriter component for managing blinded message state
- Implement proper rollback mechanisms in swap operations
- Add delete_blinded_messages database interface for cleanup
- Improve error handling with better state management
- write generated migration files to OUT_DIR instead of source directory
- copy migration SQL files to OUT_DIR for inclusion in build artifacts
- use absolute paths from OUT_DIR in include_str! macros
- update consumer modules to include from OUT_DIR using concat! macro
these changes enable cdk to support nix sandbox builds
* Add consistent ordering of sql migrations
Also sort the prefix and not only the filenames
* Reorganize tests, add mint quote/payment coverage, and prevent over-issuing
Reorganizes the mint test suite into clear modules, adds comprehensive mint
quote & payment scenarios, enhances the shared test macro, and hardens SQL
logic to forbid issuing more than what’s been paid.
These tests were added:
* Add quote once; reject duplicates.
* Register multiple payments and verify aggregated amount_paid.
* Read parity between DB and in-TX views.
* Reject duplicate payment_id in same and different transactions.
* Reject over-issuing (same TX, different TX, with/without prior payments).
---------
Co-authored-by: thesimplekid <tsk@thesimplekid.com>
- Add quote_id field to Transaction struct in cdk-common
- Add database migrations for quote_id column in SQLite and PostgreSQL
- Update wallet operations to populate quote_id for mint/melt transactions
- Set quote_id to None for send/receive operations without associated quotes
* Fix migrations ordering
1. Fix postgres migration prefixes for unreleased migrations
2. Fix build script to try to sort, if provided, the filename prefix as a
number, otherwise fallback to the sorting by filename as string
* Fixed clippy issues
* feat(cdk): add generic key-value store functionality for mint databases
Implements a comprehensive KV store system with transaction support,
namespace-based organization, and validation for mint databases.
- Add KVStoreDatabase and KVStoreTransaction traits for generic storage
- Include namespace and key validation with ASCII character restrictions
- Add database migrations for kv_store table in SQLite and PostgreSQL
- Implement comprehensive test suite for KV store functionality
- Integrate KV store traits into existing Database and Transaction bounds
Add payment_method field to MeltQuote struct to track whether quotes use BOLT11 or BOLT12 payment methods. Include database migrations for both SQLite and PostgreSQL to support the new field. Update melt operations to set and use the payment method for proper routing between BOLT11 and BOLT12 endpoints.
* feat: add LDK Node Lightning backend with comprehensive integration
- Add new cdk-ldk-node crate implementing Lightning backend using LDK Node
- Extend MintPayment trait with start/stop methods for processor lifecycle management
- Add LDK Node configuration support to cdk-mintd with chain source and gossip options
- Enhance mint startup/shutdown to properly manage payment processor lifecycle
---------
Co-authored-by: Erik <78821053+swedishfrenchpress@users.noreply.github.com>
* refactor(cdk): defer BOLT12 invoice fetching to payment execution
Move BOLT12 invoice generation from quote creation to payment time, make
request_lookup_id optional for offers, and simplify payment structures by
removing unnecessary boxing and intermediate storage of invoices.
* Add PostgreSQL support for mint and wallet
* Fixed bug to avoid empty calls `get_proofs_states`
* Fixed SQL bug
* Avoid redudant clone()
* Add more tests for the storage layer
* Minor enhacements
* Add a generic function to execute db operations
This function would log slow operations and log errors
* Provision a postgres db for tests
* Update deps for msrv
* Add postgres to pipeline
* feat: add psgl to example and docker
* feat: db url fmt
---------
Co-authored-by: thesimplekid <tsk@thesimplekid.com>
The codebase was used to correctly perform signatory calls during a database
transaction, as the signatory was previously exclusively in process. However, a
few months ago, it was changed to be a trait that can be either local or
remote. Making external calls to services, adding latency, during an ongoing
database transaction is a bad idea because it will lock the rows until the
service call is finalized, which is unpredictable.
The issue is even worse in our pipeline where the SQLite storage driver is used
with the ":memory:" path, which forces the Database pool to have a size of 1.
Since our tests run in parallel, they would randomly fail.
This issue was failing in the CI, but the error was not making the pipeline
fail. This bug was fixed as well.
* feat: refresh keysets
* fix(cdk): resolve keyset counter skipping index 0 in deterministic secret generation
- Modified Database::get_keyset_counter to return u32 instead of Option<u32>
- Added database migrations to increment existing keyset counters by 1
- Removed counter increment logic from wallet operations to use actual counter value
- Ensures deterministic secret generation starts from index 0 instead of skipping it
These functions are designed as a single funnel to talk to the database,
whether it is synchronous or asynchronous.
This single funnel will log SQL queries and slow operations, providing a clear
and unified debug message for the problematic query, so it can be optimized
accordingly (for instance, missing indexes or unbound SQL requests).
* Working on a better database abstraction
After [this question in the chat](https://matrix.to/#/!oJFtttFHGfnTGrIjvD:matrix.cashu.space/$oJFtttFHGfnTGrIjvD:matrix.cashu.space/$I5ZtjJtBM0ctltThDYpoCwClZFlM6PHzf8q2Rjqmso8)
regarding a database transaction within the same function, I realized a few
design flaws in our SQL database abstraction, particularly regarding
transactions.
1. Our upper abstraction got it right, where a transaction is bound with `&mut
self`, so Rust knows how to handle its lifetime with' async/await'.
2. The raw database does not; instead, it returns &self, and beginning a
transaction takes &self as well, which is problematic for Rust, but that's not
all. It is fundamentally wrong. A transaction should take &mut self when
beginning a transaction, as that connection is bound to a transaction and
should not be returned to the pool. Currently, that responsibility lies with
the implementor. If a mistake is made, a transaction could be executed in two
or more connections.
3. The way a database is bound to our store layer is through a single struct,
which may or may not internally utilize our connection pool. This is also
another design flow, in this PR, a connection pool is owned, and to use a
connection, it should be requested, and that connection is reference with
mutable when beginning a transaction
* Improve the abstraction with fewer generics
As suggested by @thesimplekid
* Add BEGIN IMMEDIATE for SQLite
* feat: add keyset u32 mapping migration and duplicate handling
- Add new database migration (version 3) to include u32 representation for keysets
- Implement migration for both redb and SQL databases
- Add duplicate detection and handling for keyset entries
- Create unique index constraint for keyset_u32 column in SQL
- Update keyset storage to include u32 identifiers
- Handle backwards compatibility for existing databases
* chore: clippy
* refactor(cashu): simplify keyset ID verification logic
- Consolidate match expression into a single expression
- Use direct comparison with ensure_cdk macro
- Improve readability of keyset ID validation
* refactor(cdk): rename `fetch_keyset_keys` to `load_keyset_keys` for clarity
- Renamed `fetch_keyset_keys` to `load_keyset_keys` across multiple modules to better reflect its behavior of loading keys from local storage or fetching from mint when missing.
- Added debug logging to indicate when keys are being fetched from the mint.
- Simplified key loading logic in `update_mint_keysets` by removing redundant existence checks.
* chore: remove unused vec
