* Add PostgreSQL support for mint and wallet
* Fixed bug to avoid empty calls `get_proofs_states`
* Fixed SQL bug
* Avoid redudant clone()
* Add more tests for the storage layer
* Minor enhacements
* Add a generic function to execute db operations
This function would log slow operations and log errors
* Provision a postgres db for tests
* Update deps for msrv
* Add postgres to pipeline
* feat: add psgl to example and docker
* feat: db url fmt
---------
Co-authored-by: thesimplekid <tsk@thesimplekid.com>
The codebase was used to correctly perform signatory calls during a database
transaction, as the signatory was previously exclusively in process. However, a
few months ago, it was changed to be a trait that can be either local or
remote. Making external calls to services, adding latency, during an ongoing
database transaction is a bad idea because it will lock the rows until the
service call is finalized, which is unpredictable.
The issue is even worse in our pipeline where the SQLite storage driver is used
with the ":memory:" path, which forces the Database pool to have a size of 1.
Since our tests run in parallel, they would randomly fail.
This issue was failing in the CI, but the error was not making the pipeline
fail. This bug was fixed as well.
* feat(cdk-integration-tests): refactor regtest setup and mintd integration
- Replace shell-based regtest setup with Rust binary (start_regtest_mints)
- Add cdk-mintd crate to workspace and integration tests
- Improve environment variable handling for test configurations
- Update integration tests to use proper temp directory management
- Remove deprecated start_regtest.rs binary
- Enhance CLN client connection with retry logic
- Simplify regtest shell script (itests.sh) to use new binary
- Fix tracing filters and improve error handling in setup
- Update dependencies and configurations for integration tests
fix: killing
chore: comment tests for ci debugging
chore: compile
Revert "chore: comment tests for ci debugging"
This reverts commit bfc594c11cf37caeaa6445cb854ae5567d2da6bd.
* chore: sql cipher
* fix: removal of sqlite cipher
* fix: auth password
* refactor(cdk-mintd): improve database password handling and function signatures
- Pass database password as parameter instead of parsing CLI args in setup_database
- Update function signatures for run_mintd and run_mintd_with_shutdown to accept db_password
- Remove direct CLI parsing from database setup logic
- Fix auth database initialization to use correct type when sqlcipher feature enabled
The primary purpose of this new crate is to have a common and shared codebase
for all SQL storage systems. It would force us to write standard SQL using best
practices for all databases.
This crate has been extracted from #878
- Add background service management with graceful shutdown handling
- Refactor invoice payment monitoring to use centralized task supervision
- Update cdk-mintd and integration tests to use new lifecycle methods
- Add comprehensive documentation and lifecycle tests
* Split the database trait into read and transactions.
The transaction traits will encapsulate all database changes and also expect
READ-and-lock operations to read and lock records from the database for
exclusive access, thereby avoiding race conditions.
The Transaction trait expects a `rollback` operation on Drop unless the
transaction has been committed.
* fix: melt quote duplicate error
This change stops a second melt quote from being created
if there is an existing valid melt quote for an invoice already.
If the first melt quote has expired then we allow for a new melt quote to be created.
---------
Co-authored-by: thesimplekid <tsk@thesimplekid.com>
* WIP: Introduce a SignatoryManager service.
The SignatoryManager manager provides an API to interact with keysets, private
keys, and all key-related operations, offering segregation between the mint and
the most sensible part of the mind: the private keys.
Although the default signatory runs in memory, it is completely isolated from
the rest of the system and can only be communicated through the interface
offered by the signatory manager. Only messages can be sent from the mintd to
the Signatory trait through the Signatory Manager.
This pull request sets the foundation for eventually being able to run the
Signatory and all the key-related operations in a separate service, possibly in
a foreign service, to offload risks, as described in #476.
The Signatory manager is concurrent and deferred any mechanism needed to handle
concurrency to the Signatory trait.
* Fixed missing default feature for signatory
* Do not read keys from the DB
* Removed KeysDatabase Trait from MintDatabase
All Keys operations should be done through the signatory
* Make sure signatory has all the keys in memory
Drop also foreign constraints on sqlite
* Fix race condition
* Adding debug info to failing test
* Add `sleep` in test
* Fixed issue with active auth keyset
* Fixed dependency
* Move all keys and keysets to an ArcSwap.
Since the keys and keysets exist in RAM, most wrapping functions are infallible
and synchronous, improving performance and adding breaking API changes.
The signatory will provide this information on the boot and update when the
`rotate_keyset` is executed.
Todo: Implement a subscription key to reload the keys when the GRPC server
changes the keys. For the embedded mode, that makes no sense since there is a
single way to rotate keys, and that bit is already covered.
* Implementing https://github.com/cashubtc/nuts/pull/250
* Add CLI for cdk-signatory to spawn an external signatory
Add to the pipeline the external signatory
* Update tests
* Apply suggestions from code review
Co-authored-by: ok300 <106775972+ok300@users.noreply.github.com>
Co-authored-by: thesimplekid <tsk@thesimplekid.com>
* Minor change
* Update proto buf to use the newest format
* Rename binary
* Add instrumentations
* Add more comments
* Use a single database for the signatory
Store all keys, even auth keys, in a single database. Leave the MintAuthDatabse
trait implementation for the CDK but not the signagtory
This commit also moves the cli mod to its own file
* Update dep
* Add `test_mint_keyset_gen` test
---------
Co-authored-by: ok300 <106775972+ok300@users.noreply.github.com>
Co-authored-by: thesimplekid <tsk@thesimplekid.com>
The bug comes with the SQLx-sqlite pool bug, where several connections are
created by default, but the `new` function takes care of that, fixing that bug
by making a single instance of the database.
If constructed directly, the pool would create several connections to the
database, which in most instances is fine, but with SQLite :memory: each
connection is entirely independent.
Also follow documentation to make sure that failed `acquire` will not end up
dropping connections by setting test_before_acquire to false
However, if your workload is sensitive to dropped connections such as using an in-memory
SQLite database with a pool size of 1, you can pretty easily ensure that a cancelled
`acquire()` call will never drop connections by tweaking your [`PoolOptions`]:
* Set [`test_before_acquire(false)`][PoolOptions::test_before_acquire]
* Never set [`before_acquire`][PoolOptions::before_acquire] or
[`after_connect`][PoolOptions::after_connect].
* chore: Add nutshell wallet integration test script
feat: Add MINT_URL configuration for docker container in nutshell wallet integration test script
chore: Make nutshell_wallet_itest.sh executable
fix: Update MINT_URL to use host.docker.internal for Docker container access
feat: Add Docker container startup for Cashu daemon in wallet integration test script
chore: Update Docker image to use Docker Hub repository
feat: Add cleanup trap to stop Docker container and unset variables
feat: Add initial test stub for nutshell wallet mint functionality
test: Add Cashu wallet mint integration test
feat: Add just command for nutshell wallet integration test
refactor: Organize imports and improve code formatting in nutshell wallet test
fix: Update Cashu Docker image and test URL for correct container access
fix: Update Docker container name and image for Cashu wallet test script
fix: Handle Docker container name conflict in nutshell wallet integration test
fix: Update Docker image to cashubtc/nutshell:latest in wallet integration test script
feat: Add support for running separate Nutshell mint and wallet containers
feat: Update Nutshell mint and wallet container configurations for integration testing
fix: Update wallet port and container configuration in integration test script
chore: Export MINT_URL and WALLET_URL as environment variables
fix: Update invoice creation and state checking in nutshell wallet test
fix: Update MINT_URL to use host.docker.internal for container access
fix: Update nutshell wallet mint test to handle invoice payment state
refactor: Improve Nutshell wallet test with better error handling and robustness
refactor: Improve code formatting and logging in nutshell wallet test
refactor: Replace anyhow with expect for error handling in Nutshell wallet test
refactor: Simplify error handling in Nutshell wallet mint test
refactor: Replace `?` with `expect()` in Nutshell wallet test
refactor: Simplify nutshell wallet test by removing unused code and improving error handling
refactor: Extract minting and balance helper functions in nutshell wallet test
feat: Add test for Nutshell wallet token swap functionality
fix: Trim quotes from token in nutshell wallet swap test
refactor: Remove debug print statements and improve code readability
refactor: Improve test_nutshell_wallet_melt with payment state checking and balance verification
refactor: Update Nutshell wallet integration test script configuration
feat: Extract common mint startup function into shared script
refactor: Simplify nutshell wallet integration test script and improve startup process
feat: Add mintd process termination and test status capture in integration test script
refactor: Capitalize env vars and ensure comprehensive cleanup in trap
feat: nutshell wallet test
* ci: Add test step for Nutshell wallet integration tests
* ci: Split Nutshell integration tests into separate jobs
* feat: nutshell wallet test
* ci: Add Docker setup and increase timeout for Nutshell wallet integration tests
* chore: Improve Nutshell wallet integration test script robustness
* fix: Remove -i flag from Nix develop and improve Docker accessibility check
* fix: payment processor
* fix: wallet tests
* feat: Add integration shell with Docker and Rust stable for testing
* ci: Simplify Nutshell wallet integration test workflow and script
* fix: Improve mintd endpoint detection and error handling in integration test script
* fix: wallet tests
* Here's a commit message for this change:
refactor: Move mint tests to fake_wallet.rs and add descriptive comments
refactor: pure wallet/mint does not need arc
refactor: Consolidate NUT-06 test into single function and remove redundant module
docs: Add comments explaining test purposes in integration tests file
refactor: Remove anyhow and replace with expect for error handling
refactor: use expect in pure tests
feat: Add configurable database type via environment variable for test mint and wallet
refactor: Update database initialization in test mint and wallet creation
feat: Add temporary directory support for redb and sqlite databases in tests
feat: Add database type argument to test commands in justfile
ci: Add build matrix for pure-itest with memory, sqlite, and redb databases
refactor: use expect in pure tests
refactor: Move and refactor `test_swap_unbalanced` from mint to integration tests pure
refactor: move mint tests to pure tests
docs: Add detailed comments explaining test file purposes for mint and integration tests
refactor: Extract keyset ID retrieval into a reusable function
test: Add concurrent double-spend test with 3 swap transactions
refactor: Simplify concurrent swap request processing and error handling
test: Add check to verify all proofs are marked as spent in concurrent double-spend test
refactor: Optimize proof state retrieval in concurrent double-spend test
feat: Add test for concurrent melt race condition with same proofs
fix: Update concurrent melt test to use melt quote and handle errors
refactor: melt concurrrent
refactor: Rename test function for clarity in concurrent double-spend scenario
refactor: Modify test_concurrent_double_spend_melt to manually create melt requests in mint tasks
feat: con melt test
refactor: Optimize proof state handling and error recovery in check_spendable
refactor: Extract helper method to reset proofs to original state
fix: reset y states
fix: reset y states
* fix: acces of priv feilds
* fix: add extra migrate
- MintKeysDatabase
- MintQuotesDatabase
- MintProofsDatabase
- MintSignaturesDatabase
This commit splits the MintDatabase trait with 30+ methods into a series
of smaller traits, each dedicate to a specific subsystem of the mint
service.
* Drop the in-memory database
Fixes#607
This PR drops the implementation of in-memory database traits.
They are useful for testing purposes since the tests should test our codebase
and assume the database works as expected (although a follow-up PR should write
a sanity test suite for all database trait implementors).
As complexity is worth with database requirements to simplify complexity and
add more robustness, for instance, with the following plans to add support for
transactions or buffered writes, it would become more complex and
time-consuming to support a correct database trait. This PR drops the
implementation and replaces it with a SQLite memory instance
* Remove OnceCell<Mint>
Without this change, a single Mint is shared for all tests, and the first tests
to run and shutdown makes the other databases (not-reachable, as dropping the
tokio engine would also drop the database instance).
There is no real reason, other than perhaps performance. The mint should
perhaps run in their own tokio engine and share channels as API interfaces, or
a new instance should be created in each tests
* Fixed bug with foreign keys
[1] https://gist.github.com/crodas/bad00997c63bd5ac58db3c5bd90747ed
* Show more debug on failure
* Remove old code
* Remove old references to WalletMemoryDatabase
