Commit Graph

142 Commits

Author SHA1 Message Date
callebtc
cf4cbbe882 Mint: migration to clean up overly large witnesses (#817)
* add migration to clean up overly large witnesses

* bump to 0.18.1
2025-11-04 19:39:16 +01:00
callebtc
c8566437f9 bump to 0.18.0 (#812) 2025-10-28 12:48:36 +01:00
lollerfirst
ff3fdd5aef feat: implement exponential backoff for paid_invoices_stream (#778)
* feat: implement exponential backoff for paid_invoices_stream across all Lightning backends

- Add exponential backoff retry logic to CLN REST, LND REST, and LND gRPC backends
- Start with 1 second delay, exponentially increase up to 5 minutes maximum
- Reset delay to 1 second on successful reconnection
- Improve error logging to include retry delay information
- Replace fixed delays with adaptive backoff to handle network issues gracefully
- Prevents system overload during persistent connection problems

Resolves issues with rapid reconnection attempts that could overwhelm Lightning nodes during network instability.

* remove unused import

* feat: extend exponential backoff to all remaining backends and invoice listener

- Implement exponential backoff in LNbits paid_invoices_stream for both SSE and WebSocket modes
- Add exponential backoff guidance comments to Blink and Strike backends (not implemented)
- Apply exponential backoff to invoice_listener in tasks.py that calls paid_invoices_stream
- Ensure consistent retry behavior across all Lightning backend integrations
- Improve system resilience during network interruptions and backend failures

All backends and the invoice listener now use the same exponential backoff strategy:
- Start with 1 second delay, exponentially increase up to 5 minutes maximum
- Reset delay to 1 second on successful reconnection
- Enhanced error logging with retry delay information

* blink + strike remove comments

* remove hardcoded values in favor of settings

* immediate first retry
2025-09-08 16:14:19 +02:00
lollerfirst
29571287b3 Mint Management gRPC Server (#723)
* settings

* fix name settings

* management rpc

* hook up the RPC server

* working

* format

* update build script fix import error

* remove accidental commit of vscode extension data

* working ✔

* \n

* add get mint quote get melt quote

* gRPC cli update quotes commands

* update mint melt quotes from cli

* comment under get cli command group

* keyset rotation not yet implemented

* try fix

* change back contact info default to be empty list

* fix import

* add server mTLS

* ll

* script for generating certificates

* rename settings

* move generation script

* do not save TTL expiry into Cache object, rather always load from settings.

* update lightning fees

* update auth limits

* auth rate limit cli

* optional arguemnts

* better error messages

* tests for db update mint/melt quotes

* start mint rpc tests

* add tos_url field to get-info grpc response

* format checks

* add types to click groups where it's needed

* tests on updating quotes

* fix tests

* skip updating mint quote state if on regtest

* test edge case

* unified test_add_remove_contact

* mark pytest-asyncio

* fix missing db argument

* hopefully no more silly errors

* fix test_db_update_mint_quote_state

* pass in the quote id string.

* add keyset rotation

* test for keyset rotation through gRPC command

* fix logger warning

* remove rotation test because it breaks other tests

* use different bolt11 invoices

* assert returned melt quote has quote

* is_postgres

* try different things

* skip if deprecated api

* format checks

* update .gitignore

* default location for certificates
2025-06-25 12:35:53 +02:00
Olindo Task
bbfb63b34e Add verbose request logging feature to wallet (#758)
* Add verbose request logging feature to wallet

* add verbose cli tests

---------

Co-authored-by: callebtc <93376500+callebtc@users.noreply.github.com>
2025-06-24 22:31:40 +02:00
callebtc
48823d673d Mint: adjust mint melt settings (#764)
* adjust mint melt settings

* .env.example

* ge=0, closes #756
2025-06-01 12:24:37 +02:00
callebtc
3e32dc40e3 add msat support for strike backend (#755) 2025-05-18 15:48:15 +02:00
callebtc
c6f7ab1e79 ledger watchdog: disable the abort queue for now (#754) 2025-05-14 13:04:28 +02:00
callebtc
081a30ea68 bump version to 0.17.0 (#751) 2025-05-13 18:01:20 +02:00
callebtc
fc0e3fe663 Mint: watchdog balance log and killswitch (#705)
* wip store balance

* store balances in watchdog worker

* move mint_auth_database setting

* auth db

* balances returned as Amount (instead of int)

* add test for balance change on invoice receive

* fix 1 test

* cancel tasks on shutdown

* watchdog can now abort

* remove wallet api server

* fix lndgrpc

* fix lnbits balance

* disable watchdog

* balance lnbits msat

* test db watcher with its own database connection

* init superclass only once

* wip: log balance in keysets table

* check max balance using new keyset balance

* fix test

* fix another test

* store fees in keysets

* format

* cleanup

* shorter

* add keyset migration to auth server

* fix fakewallet

* fix db tests

* fix postgres problems during migration 26 (mint)

* fix cln

* ledger

* working with pending

* super fast watchdog, errors

* test new pipeline

* delete walletapi

* delete unneeded files

* revert workflows
2025-05-11 20:29:13 +02:00
callebtc
3d21443c3c bump to 0.16.6 (#747) 2025-05-10 15:31:56 +02:00
callebtc
7f962ce63e Wallet: add Wallet.get_melt_proof(), recover reserved state in failed Wallet.melt() (#730)
* manage state of proofs

* test set proofs unreserved

* melt recovers state on error

* fix wallet test

* fix tests

* fix another test

* get_mint_quote returns MintQuote

* fix tests

* refactor

* fix deprecated tests

* add new unset type
2025-04-25 13:12:45 +02:00
callebtc
7abfc68cfa SIG_ALL signature flag for P2PK (#735)
* n_sigs_refund working, tests added

* update requirements

* wip sigall

* wip

* sigall works

* add signatures for refund

* add mint p2pk tests

* add more p2pk tests

* fix tests

* sign htlc pubkeys as well

* fix htlc and add new test

* fix regtest

* fix new tests with deprecated

* remove asserts

* comments

* new wallet p2pk tests

* getting there

* add more tests

* fixes

* refactor htlc and p2pk validation

* reduce code

* melt with sigall

* fix htlcs

* fix deprecated api tests

* Update cashu/mint/conditions.py

Co-authored-by: lollerfirst <43107113+lollerfirst@users.noreply.github.com>

* refactor sigall validation

---------

Co-authored-by: lollerfirst <43107113+lollerfirst@users.noreply.github.com>
2025-04-25 11:37:19 +02:00
callebtc
e1220d2329 Cjbeery24/mint copy: Invoice checker background tasks (#722)
* #616: Removed blocking call from ledger startup. Instead added it to a background task that repeats every hour.

* make task interval configurable, remember task and cancel it on shutdown

* comments

* add sleep to tests because the background task is async

---------

Co-authored-by: Caleb Beery <cjbeery@gmail.com>
2025-04-10 21:25:18 +07:00
callebtc
76e2601efd Add terms of service URL to MintInformation class (#715)
* add tos_url according to https://github.com/cashubtc/nuts/pull/205

* add tos_url everywhere
2025-03-28 16:11:04 +01:00
callebtc
a0ef44dba0 Blind authentication (#675)
* auth server

* cleaning up

* auth ledger class

* class variables -> instance variables

* annotations

* add models and api route

* custom amount and api prefix

* add auth db

* blind auth token working

* jwt working

* clean up

* JWT works

* using openid connect server

* use oauth server with password flow

* new realm

* add keycloak docker

* hopefully not garbage

* auth works

* auth kinda working

* fix cli

* auth works for send and receive

* pass auth_db to Wallet

* auth in info

* refactor

* fix supported

* cache mint info

* fix settings and endpoints

* add description to .env.example

* track changes for openid connect client

* store mint in db

* store credentials

* clean up v1_api.py

* load mint info into auth wallet

* fix first login

* authenticate if refresh token fails

* clear auth also middleware

* use regex

* add cli command

* pw works

* persist keyset amounts

* add errors.py

* do not start auth server if disabled in config

* upadte poetry

* disvoery url

* fix test

* support device code flow

* adopt latest spec changes

* fix code flow

* mint max bat dynamic

* mypy ignore

* fix test

* do not serialize amount in authproof

* all auth flows working

* fix tests

* submodule

* refactor

* test

* dont sleep

* test

* add wallet auth tests

* test differently

* test only keycloak for now

* fix creds

* daemon

* fix test

* install everything

* install jinja

* delete wallet for every test

* auth: use global rate limiter

* test auth rate limit

* keycloak hostname

* move keycloak test data

* reactivate all tests

* add readme

* load proofs

* remove unused code

* remove unused code

* implement change suggestions by ok300

* add error codes

* test errors
2025-01-29 22:48:51 -06:00
callebtc
b67ffd8705 bump to 0.16.5 (#699) 2025-01-29 15:25:31 -06:00
callebtc
5e5a1a57d9 bump to 0.16.4 (#685) 2025-01-03 22:45:16 +01:00
lollerfirst
399c201552 NUT-19: Cached Requests and Responses (#624)
* fast-api-cache setup

* testing the cache

* fix

* still not working

* asynccontextmanager

* move test

* use redis & custom caching setup (like CDK)

* make format

* poetry lock

* fix format string + log when a cached response is found

* log when a cahced response is found

* fix tests

* poetry lock

* try tests on github

* use docker compose

* maybe we dont need docker

* fix types

* create_task instead of run

* how about we start postgres

* mint features

* format

* remove deprecated setex call

* use global expiry for all cached routes

* refactor feature map and set default to 1 week

* refactor feature construction

* Cache NUT-19

---------

Co-authored-by: callebtc <93376500+callebtc@users.noreply.github.com>
2024-12-03 16:03:01 +03:00
callebtc
6a4f1bdbe9 bump version to 0.16.3 (#669) 2024-11-14 14:55:22 +01:00
callebtc
d8a90d4c80 turn on mpp by default (#667) 2024-11-05 15:40:17 +01:00
Pavol Rusnak
0de574e006 NUT-06: add urls field (#638) 2024-11-05 15:03:23 +01:00
callebtc
05547e3710 Mint: Turn off locking by default (#662)
* turn off locking by default

* fix test
2024-11-04 10:52:33 +01:00
callebtc
f98ed30483 bump version (#664) 2024-11-04 10:52:09 +01:00
callebtc
307d2d7a98 Halt melt on exception (#635)
* halt melt for payment status exception

* refactor

* less log

* add fakewallet test throwing exceptions
2024-10-06 17:51:54 +02:00
callebtc
7fdca3b1a1 Fix: ledger requires deprecated paid flags in db + wallet disable base64 keysets by default (#634)
* wallet: deprecate base64 keysets by default

* readd deprecated paid to melt quotes as well

* readd paid flag removed in #622 before
2024-10-04 15:32:36 +02:00
callebtc
4d8b6a7a74 bump version to 0.16.1 (#633) 2024-10-04 14:02:46 +02:00
callebtc
d8d3037cc5 WIP: New melt flow (#622)
* `PaymentResult`

* ledger: rely on PaymentResult instead of paid flag. Double check for payments marked pending.

* `None` is `PENDING`

* make format

* reflected changes API tests where `PaymentStatus` is used + reflected changes in lnbits

* reflect changes in blink backend and tests

* fix lnbits get_payment_status

* remove paid flag

* fix mypy

* remove more paid flags

* fix strike mypy

* green

* shorten all state checks

* fix

* fix some tests

* gimme 

* fix............

* fix lnbits

* fix error

* lightning refactor

* add more regtest tests

* add tests for pending state and failure

* shorten checks

* use match case for startup check - and remember modified checking_id from pay_invoice

* fix strike pending return

* new tests?

* refactor startup routine into get_melt_quote

* test with purge

* refactor blink

* cleanup responses

* blink: return checking_id on failure

* fix lndgrpc try except

* add more testing for melt branches

* speed things up a bit

* remove comments

* remove comments

* block pending melt quotes

* remove comments

---------

Co-authored-by: lollerfirst <lollerfirst@gmail.com>
2024-09-24 14:55:35 +02:00
callebtc
ef5aee92d6 NUT-06: add icon URL (#604)
* mint info icon

* Update cashu/core/models.py

Co-authored-by: Pavol Rusnak <pavol@rusnak.io>

* Update cashu/mint/router.py

Co-authored-by: Pavol Rusnak <pavol@rusnak.io>

* Update cashu/wallet/mint_info.py

Co-authored-by: Pavol Rusnak <pavol@rusnak.io>

* fix setting

---------

Co-authored-by: Pavol Rusnak <pavol@rusnak.io>
2024-09-08 15:00:32 +02:00
lollerfirst
d388508521 Mint: Talk to LND via gRPC (#595)
* protos + lnd_grpc.py + __init__ +  status method

* `create_invoice`

* `pay_invoice`, `pay_partial_invoice` and router pb2

* `get_invoice_status`

* channel keep-alive options

* Update lnd_grpc.py

* `get_payment_status` + make format

* `get_payment_quote` and `paid_invoices_stream`. This was suspiciously easy...

* download_and_build script modified to fix the imports on generated code

* pyproject with new dependencies

* update poetry.lock

* fixed errors in `pay_partial_invoice`

* update .env.example

* make format

* enable regtest

* update .env.example

* suggested fixes

* suggested changes pt.2

* Update cashu/core/settings.py

Co-authored-by: callebtc <93376500+callebtc@users.noreply.github.com>

---------

Co-authored-by: callebtc <93376500+callebtc@users.noreply.github.com>
2024-07-29 17:16:40 +02:00
callebtc
71580a5e60 Fix parsing of old format contact field in wallet (#589)
* wallet parse contact field if its old format

* default checks for contacts
2024-07-17 11:48:40 +02:00
callebtc
77697c52ee Fix loading b64 keysets and add option to set b64 inactive in WalletSettings (#579)
* Mint: fix loading b64 keysets and Wallet: option to set b64 inactive

* typo

* readd include fees flag (unused)

* fix test to respect new default False flag

* fix default flag for regtest
2024-07-11 15:25:16 +02:00
callebtc
eccbf6f789 bump version to 0.16.0 (#573) 2024-07-09 16:36:09 +02:00
callebtc
6a0a370ba5 Mint: table locks (#566)
* clean up db

* db: table lock

* db.table_with_schema

* fix encrypt.py

* postgres nowait

* add timeout to lock

* melt quote state in db

* kinda working

* kinda working with postgres

* remove dispose

* getting there

* porperly clean up db for tests

* faster tests

* configure connection pooling

* try github with connection pool

* invoice dispatcher does not lock db

* fakewallet: pay_if_regtest waits

* pay fakewallet invoices

* add more

* faster

* slower

* pay_if_regtest async

* do not lock the invoice dispatcher

* test: do I get disk I/O errors if we disable the invoice_callback_dispatcher?

* fix fake so it workss without a callback dispatchert

* test on github

* readd tasks

* refactor

* increase time for lock invoice disatcher

* try avoiding a race

* remove task

* github actions: test regtest with postgres

* mint per module

* no connection pool for testing

* enable pool

* do not resend paid event

* reuse connection

* close db connections

* sessions

* enable debug

* dispose engine

* disable connection pool for tests

* enable connection pool for postgres only

* clean up shutdown routine

* remove wait for lightning fakewallet lightning invoice

* cancel invoice listener tasks on shutdown

* fakewallet conftest: decrease outgoing delay

* delay payment and set postgres only if needed

* disable fail fast for regtest

* clean up regtest.yml

* change order of tests_db.py

* row-specific mint_quote locking

* refactor

* fix lock statement

* refactor swap

* refactor

* remove psycopg2

* add connection string example to .env.example

* remove unnecessary pay

* shorter sleep in test_wallet_subscription_swap
2024-07-08 18:05:57 +02:00
lollerfirst
ca272bc20f Mint: Add clnrest.py Lightning backend (#551)
* log cln error

* return a string

* update corelightningrest to work with latest ver using rune

* fix mpp spec and backend support check

* refactor validation in ledger

* remove weird error

* fix mpp melt model

* corelightningrest.py: Added Multi-Mint payout support
lndrest.py: fix `quote.amount` is not always in sats + better checks

* small fix

* Fix quote.unit str2unit conversion + add missing imports

* settings enable mpp corelightning (default false)

* small fix

* fix `paid_invoice_stream`

* make format

* handle runes

* load rune

* rename to MINT_CORELIGHTNING_REST_RUNE

* try without cert

* port

* try except callback dispatcher

* clean up cln-rest streaming parser

* conftest: mint_corelightning_enable_mpp

* enable mpp in regtest.yaml

* fix error handling clnrest, remove lndrest changes

* CLNRest + CoreLightningRest

* clean up corelightningrest and get last index before starting the stream

* clean up

---------

Co-authored-by: callebtc <93376500+callebtc@users.noreply.github.com>
Co-authored-by: Richard Bensberg <r@coinbatsu.com>
2024-06-30 20:36:19 +02:00
elliedev80
6946bed8c3 add MINT_LND_REST_CERT_VERIFY env bool that when set to False allow to skip certificate validation for LND api call (#535)
* LND Backend
 - Add MINT_LND_REST_CERT_VERIFY bool variable that when set to False allows
   verify=False for httpx and ignore LND selfsigned certificate validation

 On branch main
 Your branch is up to date with 'origin/main'.

 Changes to be committed:
	modified:   .env.example
	modified:   cashu/core/settings.py
	modified:   cashu/lightning/lndrest.py

* Update .env.example

---------

Co-authored-by: callebtc <93376500+callebtc@users.noreply.github.com>
2024-06-26 15:03:19 +02:00
callebtc
1766b6e92e [Mint] Add support for BTC and EUR in StrikeWallet backend, add EUR to FakeWallet (#561)
* strike for btc and eur

* strike works with eur

* backend check
2024-06-26 14:50:39 +02:00
callebtc
e846acf946 Mint: add websockets for quote updates (#413)
* add websockets for quote updates

* add test (not working)

* wip: emit events to everyone

* wip: emit events to everyone

* wip, lots of things broken but invoice callback works

* wip

* add wip files

* tests almost passing

* add task

* refactor nut constants

* startup fix

* works with old mints

* wip cli

* fix mypy

* remove automatic invoice test now with websockets

* remove comment

* better logging

* send back response

* add rate limiter to websocket

* add rate limiter to subscriptions

* refactor websocket ratelimit

* websocket tests

* subscription kinds

* doesnt start

* remove circular import

* update

* fix mypy

* move test file in test because it fails if it runs later... dunno why

* adjust websocket NUT-06 settings

* local import and small fix

* disable websockets in CLI if "no_check" is selected

* move subscription test to where it was

* check proof state with callback, add tests

* tests: run mint fixture per module instead of per session

* subscription command name fix

* test per session again

* update test race conditions

* fix tests

* clean up

* tmp

* fix db issues and remove cached secrets

* fix tests

* blindly try pipeline

* remove comments

* comments
2024-06-25 19:20:03 +02:00
callebtc
d30b1a2777 Add fees (#503)
* wip

* wip

* model

* refactor wallet transactions

* refactor wallet

* sending with fees works and outputs fill up the wallet

* wip work

* ok

* comments

* receive with amount=0

* correctly import postmeltrequest

* fix melt amount

* tests working

* remove mint_loaded decorator in deprecated wallet api

* wallet works with units

* refactor: melt_quote

* fix fees

* add file

* fees for melt inputs

* set default input fee for internal quotes to 0

* fix coinselect

* coin selection working

* yo

* fix all tests

* clean up

* last commit added fees for inputs for melt transactions - this commit adds a blanace too low exception

* fix fee return and melt quote max allowed amount check during creation of melt quote

* clean up code

* add tests for fees

* add melt tests

* update wallet fee information
2024-06-15 16:22:41 +02:00
callebtc
61cf7def24 Multinut LND (#492)
* amount in melt request

* apply fee limit

* more error handling

* wip: signal flag in /info

* clean up multinut

* decode mypy error lndrest

* fix test

* fix tests

* signal feature and blindmessages_deprecated

* setting

* fix blindedsignature method

* fix tests

* mint info file

* test mpp with lnd regtest

* nuts optionsl mint
 info

* try to enable mpp with lnd

* test mpp with third payment
2024-05-22 22:52:26 +02:00
callebtc
fa3e2fd130 default setting for mint_derivation_path_list (#515) 2024-04-16 17:44:36 +02:00
callebtc
bdaed8451c [Mint] Migrate duplicate keysets to db (#511)
* wip

* remove all deprecated keyset tests

* fix more tests

* fixups
2024-04-15 00:33:23 +02:00
callebtc
19de10bfea [Refactor] Mint: remove output.id optional (#504)
* remove output.id optional

* asserts

* wip

* wip

* working
2024-04-10 12:23:53 +02:00
callebtc
b8ad0e0a8f Mint: Recover pending melts at startup (#499)
* wip works with fakewallet

* startup refactor

* add tests

* regtest tests for pending melts

* wip CLN

* remove db migration

* remove foreign key relation to keyset id

* fix: get_promise from db and restore DLEQs

* test: check for keyset not found error

* fix migrations

* lower-case all db column names

* add more tests for regtest

* simlate failure for lightning

* test wallet spent state with hodl invoices

* retry

* regtest with postgres

* retry postgres

* add sleeps

* longer sleep on github

* more sleep for github sigh

* increase sleep ffs

* add sleep loop

* try something

* do not pay with wallet but with ledger

* fix lnbits pending state

* fix pipeline to use fake admin from docker
2024-04-03 17:14:21 +02:00
callebtc
4ff9e6de5d bump version to 0.15.2 (#495) 2024-03-26 22:01:09 +01:00
callebtc
adeec000a7 Mint: Add slowapi (#481)
* Add slowapi

* fix startup

* adjust settings

* add rate limits to tx routes

* elastic
2024-03-23 02:25:19 +01:00
callebtc
3ba1e81fcb Mint: Fakewallet support for USD (#488)
* fakewallet usd wip

* FakeWallet support for USD

* fix api return for receive

* use MINT_BACKEND_BOLT11_SAT everywhere
2024-03-22 12:11:40 +01:00
callebtc
ff1e7597d1 Merge both h2c tests and add tests for deterministic blinding factor generation (#466) 2024-03-04 15:34:51 +01:00
callebtc
29be002495 Wallet: deprecate old h2c (#459)
* wallet: deprecate old hash to curve

* fix order

* added migration: untested

* recompute Y always
2024-02-26 23:07:13 +01:00
callebtc
53cd8ff016 bump version to 0.15.1 (#461) 2024-02-26 01:50:37 +01:00