Commit Graph

308 Commits

Author SHA1 Message Date
callebtc
cf4cbbe882 Mint: migration to clean up overly large witnesses (#817)
* add migration to clean up overly large witnesses

* bump to 0.18.1
2025-11-04 19:39:16 +01:00
lollerfirst
4a4b7f79f7 fix: unique ('PENDING', checking_ID) for melt quotes (#800)
* fix: only one melt_quote with shared checking_id is allowed to be in a pending state.

fix mypy

add comprehensive tests

remove SQL unique index

remove test db constraint

fix lock statement

remove `test_regtest_pay_mpp_incomplete_payment`

format

* remove `test_set_melt_quote_pending_with_outputs`

* client self-rug mitigation

* fix

* format

* DB level check: error if payment reference paid or pending

* fix test

* comments

* restore

* restore

---------

Co-authored-by: callebtc <93376500+callebtc@users.noreply.github.com>
2025-10-28 11:47:23 +01:00
callebtc
184f8ba6ca Revert "extend and refactor nut11 sig_all message aggregation (#804)" (#810)
This reverts commit 7000e5c7ee.
2025-10-28 11:42:21 +01:00
callebtc
7000e5c7ee extend and refactor nut11 sig_all message aggregation (#804) 2025-10-28 11:41:44 +01:00
lollerfirst
f84028ca3f refactor: HTLC spending conditions (#803)
* refactor nut14

* format

* tests

* update tests

* more tests

* format

* adjust wallet tests -> 32 bytes preimages

* format
2025-10-21 16:37:51 +02:00
callebtc
23683f5985 check for existing b (#802) 2025-10-20 11:53:13 +02:00
callebtc
9fed0f0f07 Fix duplicate blank outputs during melt (#795)
* wip blank outputs

* wip: working

* store ids for promises correctly

* tests

* fix migraiton

* revert

* fix tests

* fix auth server

* fix last tests

* retroactively change migration, initial and m017_foreign_keys_proof_tables, remove c_b and replace with c_ (same for b_)

* fix constraint

* oops

* msg stack fix

* fix test foreign key constraint

* fix postgres tests

* foreign key constraint test

* should fix psql error

* foreign key constraint sqlite

* rename to update_blinded_message_signature

* drop outputs and change columns from melt_quotes table

* switch migration order

* reorder migrations again

* fix migration

* add tests

* fix postgres migration too

* create signed_at column postgres

* foreign key constraingt promises table

* migration tool

* readme
2025-10-19 15:50:47 +02:00
callebtc
15ec97b9f7 add description to NUT-04 method options (#783)
* add description to NUT-04 method options

* add option
2025-09-09 22:30:42 +02:00
lollerfirst
3b4f5b56a0 fix: subscription re-init bug (#781)
* fix subscription re-initialization bug

* test for regression issue

* format

* Update cashu/mint/events/client.py

Co-authored-by: callebtc <93376500+callebtc@users.noreply.github.com>

---------

Co-authored-by: callebtc <93376500+callebtc@users.noreply.github.com>
2025-09-08 18:20:21 +02:00
lollerfirst
ff3fdd5aef feat: implement exponential backoff for paid_invoices_stream (#778)
* feat: implement exponential backoff for paid_invoices_stream across all Lightning backends

- Add exponential backoff retry logic to CLN REST, LND REST, and LND gRPC backends
- Start with 1 second delay, exponentially increase up to 5 minutes maximum
- Reset delay to 1 second on successful reconnection
- Improve error logging to include retry delay information
- Replace fixed delays with adaptive backoff to handle network issues gracefully
- Prevents system overload during persistent connection problems

Resolves issues with rapid reconnection attempts that could overwhelm Lightning nodes during network instability.

* remove unused import

* feat: extend exponential backoff to all remaining backends and invoice listener

- Implement exponential backoff in LNbits paid_invoices_stream for both SSE and WebSocket modes
- Add exponential backoff guidance comments to Blink and Strike backends (not implemented)
- Apply exponential backoff to invoice_listener in tasks.py that calls paid_invoices_stream
- Ensure consistent retry behavior across all Lightning backend integrations
- Improve system resilience during network interruptions and backend failures

All backends and the invoice listener now use the same exponential backoff strategy:
- Start with 1 second delay, exponentially increase up to 5 minutes maximum
- Reset delay to 1 second on successful reconnection
- Enhanced error logging with retry delay information

* blink + strike remove comments

* remove hardcoded values in favor of settings

* immediate first retry
2025-09-08 16:14:19 +02:00
lollerfirst
e361cff5c1 management RPC follow up (#770)
* new protobuf file

* redis

* format
2025-08-05 12:52:00 +02:00
lollerfirst
92bf6311fb [BUGFIX] SIG_ALL check returns true for other sigflags (#767)
* solved the issue

* revised approach

* fix

* fix 2
2025-08-05 12:51:37 +02:00
lollerfirst
29571287b3 Mint Management gRPC Server (#723)
* settings

* fix name settings

* management rpc

* hook up the RPC server

* working

* format

* update build script fix import error

* remove accidental commit of vscode extension data

* working ✔

* \n

* add get mint quote get melt quote

* gRPC cli update quotes commands

* update mint melt quotes from cli

* comment under get cli command group

* keyset rotation not yet implemented

* try fix

* change back contact info default to be empty list

* fix import

* add server mTLS

* ll

* script for generating certificates

* rename settings

* move generation script

* do not save TTL expiry into Cache object, rather always load from settings.

* update lightning fees

* update auth limits

* auth rate limit cli

* optional arguemnts

* better error messages

* tests for db update mint/melt quotes

* start mint rpc tests

* add tos_url field to get-info grpc response

* format checks

* add types to click groups where it's needed

* tests on updating quotes

* fix tests

* skip updating mint quote state if on regtest

* test edge case

* unified test_add_remove_contact

* mark pytest-asyncio

* fix missing db argument

* hopefully no more silly errors

* fix test_db_update_mint_quote_state

* pass in the quote id string.

* add keyset rotation

* test for keyset rotation through gRPC command

* fix logger warning

* remove rotation test because it breaks other tests

* use different bolt11 invoices

* assert returned melt quote has quote

* is_postgres

* try different things

* skip if deprecated api

* format checks

* update .gitignore

* default location for certificates
2025-06-25 12:35:53 +02:00
callebtc
48823d673d Mint: adjust mint melt settings (#764)
* adjust mint melt settings

* .env.example

* ge=0, closes #756
2025-06-01 12:24:37 +02:00
callebtc
3e32dc40e3 add msat support for strike backend (#755) 2025-05-18 15:48:15 +02:00
callebtc
c6f7ab1e79 ledger watchdog: disable the abort queue for now (#754) 2025-05-14 13:04:28 +02:00
callebtc
fc0e3fe663 Mint: watchdog balance log and killswitch (#705)
* wip store balance

* store balances in watchdog worker

* move mint_auth_database setting

* auth db

* balances returned as Amount (instead of int)

* add test for balance change on invoice receive

* fix 1 test

* cancel tasks on shutdown

* watchdog can now abort

* remove wallet api server

* fix lndgrpc

* fix lnbits balance

* disable watchdog

* balance lnbits msat

* test db watcher with its own database connection

* init superclass only once

* wip: log balance in keysets table

* check max balance using new keyset balance

* fix test

* fix another test

* store fees in keysets

* format

* cleanup

* shorter

* add keyset migration to auth server

* fix fakewallet

* fix db tests

* fix postgres problems during migration 26 (mint)

* fix cln

* ledger

* working with pending

* super fast watchdog, errors

* test new pipeline

* delete walletapi

* delete unneeded files

* revert workflows
2025-05-11 20:29:13 +02:00
lollerfirst
619d06f0ab [PATCH] LND use_mission_control + exclude failing channels (#738)
* lnd_grpc multinut patch

* lndrest multinut patch

* mypy fixes

* fixes non escaped double quotes in error messages formats

* fix

* fix debug log with correct hops number

* correctly escape "hops"

* remove `ignored_pairs` constraint

* Apply suggestions from code review

change some error logs to debug

* add tests and some cleanup

---------

Co-authored-by: callebtc <93376500+callebtc@users.noreply.github.com>
2025-05-10 15:45:15 +02:00
callebtc
2135b96706 Testing: check if pipeline gets stuck because of startup task (#745)
* lets check if this gets stuck with postgres or not

* check if it works if I add sleeps

* replace startup_ledger with await ledger._check_pending_proofs_and_melt_quotes() in tests

* fix typo to trigger tests again
2025-05-10 15:04:56 +02:00
lollerfirst
213d5b1eee Mint Keyset Rotation (#734)
* move keysets methods into newly created `LedgerKeysets` + `maybe_update_derivation_path` on init

* keyset rotation

* fix error

* make format

* format

* test keyset rotation + fix bug

* format

* fix for multiple specified derivation paths

* specify return value
2025-04-25 11:37:49 +02:00
callebtc
7abfc68cfa SIG_ALL signature flag for P2PK (#735)
* n_sigs_refund working, tests added

* update requirements

* wip sigall

* wip

* sigall works

* add signatures for refund

* add mint p2pk tests

* add more p2pk tests

* fix tests

* sign htlc pubkeys as well

* fix htlc and add new test

* fix regtest

* fix new tests with deprecated

* remove asserts

* comments

* new wallet p2pk tests

* getting there

* add more tests

* fixes

* refactor htlc and p2pk validation

* reduce code

* melt with sigall

* fix htlcs

* fix deprecated api tests

* Update cashu/mint/conditions.py

Co-authored-by: lollerfirst <43107113+lollerfirst@users.noreply.github.com>

* refactor sigall validation

---------

Co-authored-by: lollerfirst <43107113+lollerfirst@users.noreply.github.com>
2025-04-25 11:37:19 +02:00
callebtc
e1220d2329 Cjbeery24/mint copy: Invoice checker background tasks (#722)
* #616: Removed blocking call from ledger startup. Instead added it to a background task that repeats every hour.

* make task interval configurable, remember task and cancel it on shutdown

* comments

* add sleep to tests because the background task is async

---------

Co-authored-by: Caleb Beery <cjbeery@gmail.com>
2025-04-10 21:25:18 +07:00
callebtc
76e2601efd Add terms of service URL to MintInformation class (#715)
* add tos_url according to https://github.com/cashubtc/nuts/pull/205

* add tos_url everywhere
2025-03-28 16:11:04 +01:00
callebtc
f783493e1a Fix mint migrations: Fill in missing keyset IDs in proofs and promises (#729)
* mint migrations: fill in missing keyset ids in proofs and promises

* do not throw exception

* add error where proofs are missing an id but no keyset was found

* fix balance view initialization
2025-03-27 11:33:20 +01:00
callebtc
bae4855915 NUT-04/05: add amount, unit, request to melt and mint quote responses (#719)
* add amount, unit, request to melt and mint responses

* make new fields optional to not break compat with old mints

* make new flags optional for backwards compat
2025-03-08 18:23:34 +00:00
callebtc
9305905a85 Add LightningPaymentFailedError exception (#706)
* add error code

* fix test
2025-03-06 00:04:37 +01:00
lollerfirst
f72a3f260f [FIX] NUT-15 mpp amount in millisats (#703)
* fix lndrest

* fix clnrest

* fix clnrest and lndrest

* lnd grpc fix

* wallet

* convert amount to millisats in CLI pay invoice

* fix tests

* format

* fix kw arg in regtest test

* fix payment quote validation check

* clean comment

* avoid overwriting variable

* deprecated response with amount

---------

Co-authored-by: callebtc <93376500+callebtc@users.noreply.github.com>
2025-03-05 23:47:03 +01:00
lollerfirst
0c40bbbc08 Balance Views Grouped By Keyset (#652)
* balance view per keyset + relative changes.
    Still lacking: settings changes

* import unit

* fix 0 balance

* settings

---------

Co-authored-by: callebtc <93376500+callebtc@users.noreply.github.com>
2025-02-17 12:55:28 +01:00
callebtc
23a706d873 Set correct blink min fee (#701)
* min fee

* fix tests

* amount_split test
2025-02-13 15:53:26 +01:00
lollerfirst
ec3db81cee Update Error Codes (#702)
* add new error codes

* use the new errors in the code

* fix unsorted import

* fix test wallet
2025-02-13 15:35:06 +01:00
callebtc
a0ef44dba0 Blind authentication (#675)
* auth server

* cleaning up

* auth ledger class

* class variables -> instance variables

* annotations

* add models and api route

* custom amount and api prefix

* add auth db

* blind auth token working

* jwt working

* clean up

* JWT works

* using openid connect server

* use oauth server with password flow

* new realm

* add keycloak docker

* hopefully not garbage

* auth works

* auth kinda working

* fix cli

* auth works for send and receive

* pass auth_db to Wallet

* auth in info

* refactor

* fix supported

* cache mint info

* fix settings and endpoints

* add description to .env.example

* track changes for openid connect client

* store mint in db

* store credentials

* clean up v1_api.py

* load mint info into auth wallet

* fix first login

* authenticate if refresh token fails

* clear auth also middleware

* use regex

* add cli command

* pw works

* persist keyset amounts

* add errors.py

* do not start auth server if disabled in config

* upadte poetry

* disvoery url

* fix test

* support device code flow

* adopt latest spec changes

* fix code flow

* mint max bat dynamic

* mypy ignore

* fix test

* do not serialize amount in authproof

* all auth flows working

* fix tests

* submodule

* refactor

* test

* dont sleep

* test

* add wallet auth tests

* test differently

* test only keycloak for now

* fix creds

* daemon

* fix test

* install everything

* install jinja

* delete wallet for every test

* auth: use global rate limiter

* test auth rate limit

* keycloak hostname

* move keycloak test data

* reactivate all tests

* add readme

* load proofs

* remove unused code

* remove unused code

* implement change suggestions by ok300

* add error codes

* test errors
2025-01-29 22:48:51 -06:00
lollerfirst
e9952e59ed [FIX] Reject Internal MPP Melt Quote Requests (#697)
* fix

* fix the fix

* make sure the mpp is supported before testing

* adjust message

---------

Co-authored-by: callebtc <93376500+callebtc@users.noreply.github.com>
2025-01-29 14:00:35 -06:00
lollerfirst
ea96fab9e3 [FIX] Specs conformant error codes (#693)
* error codes must follow spec

* make format
2025-01-21 17:36:22 -06:00
callebtc
4e7917fb20 Fix Redis teardown logic based on usage (#696)
* fix: tear down redis only if used

* fix: close redis only if its used
2025-01-21 17:35:24 -06:00
callebtc
ad7c6b8e0b Issue NUT-08 overpaid Lightning fees for melt quote checks on startup (#688)
* startup: do not rollback unknown melt quote states

* fix: provide overpaid fees on startup

* fix: check if outputs in db

* fix test: expect melt quote pending if payment state is unknown

* fix up comment
2025-01-21 17:28:41 -06:00
lollerfirst
d98d166df1 Support NUT-XX (signatures on quotes) for mint and wallet side (#670)
* nut-19 sign mint quote

* ephemeral key for quote

* `mint` adjustments + crypto/nut19.py

* wip: mint side working

* fix import

* post-merge fixups

* more fixes

* make format

* move nut19 to nuts directory

* `key` -> `privkey` and `pubkey`

* make format

* mint_info method for nut-19 support

* fix tests imports

* fix signature missing positional argument + fix db migration format not correctly escaped + pass in NUT-19 keypair to `request_mint` `request_mint_with_callback`

* make format

* fix `get_invoice_status`

* rename to xx

* nutxx -> nut20

* mypy

* remove `mint_quote_signature_required` as per spec

* wip edits

* clean up

* fix tests

* fix deprecated api tests

* fix redis tests

* fix cache tests

* fix regtest mint external

* fix mint regtest

* add test without signature

* test pubkeys in quotes

* wip

* add compat

---------

Co-authored-by: callebtc <93376500+callebtc@users.noreply.github.com>
2024-12-15 00:39:53 +01:00
lollerfirst
399c201552 NUT-19: Cached Requests and Responses (#624)
* fast-api-cache setup

* testing the cache

* fix

* still not working

* asynccontextmanager

* move test

* use redis & custom caching setup (like CDK)

* make format

* poetry lock

* fix format string + log when a cached response is found

* log when a cahced response is found

* fix tests

* poetry lock

* try tests on github

* use docker compose

* maybe we dont need docker

* fix types

* create_task instead of run

* how about we start postgres

* mint features

* format

* remove deprecated setex call

* use global expiry for all cached routes

* refactor feature map and set default to 1 week

* refactor feature construction

* Cache NUT-19

---------

Co-authored-by: callebtc <93376500+callebtc@users.noreply.github.com>
2024-12-03 16:03:01 +03:00
ok300
ee90d840ab Add HTTP compression middleware (#676)
* Add HTTP compression middleware

* Apply fixes from `make format`
2024-11-25 13:08:46 +01:00
callebtc
2b233fd67e fix: NUT-15 setting remove mpp boolean (#673)
* fix: NUT-15 setting remove mpp boolean

* remove mpp flag
2024-11-24 16:55:17 +01:00
callebtc
0a230a1aa3 catch websocket disconenct errors (#674) 2024-11-24 16:48:39 +01:00
callebtc
901b16713c add methods key to info endpoitn a la https://github.com/cashubtc/nuts/pull/190 (#672) 2024-11-15 16:21:54 +01:00
Pavol Rusnak
0de574e006 NUT-06: add urls field (#638) 2024-11-05 15:03:23 +01:00
callebtc
602687b215 refactor pubkey extraction and set n_sigs=1 for refund spend path (#644) 2024-11-05 15:02:54 +01:00
elnosh
9cdfba52a3 add nut-14 as supported in info endpoint (#647) 2024-11-05 13:52:09 +01:00
callebtc
05547e3710 Mint: Turn off locking by default (#662)
* turn off locking by default

* fix test
2024-11-04 10:52:33 +01:00
callebtc
e9d4689b4b Chore: update dev dependencies (#658)
* update dev dependencies

* test

* revert tests

* fix event loop scope

* ruff format

* add old test

* remove custom event loop

* default loop scope

* skip earlier

* trigger

* trigger again
2024-11-01 15:07:56 +01:00
callebtc
92627399a5 Wallet: store quotes (#657)
* wallet_quotes_wip

* fix quote in db

* fix subscription test

* clean up api

* fix api tests

* fix balance check
2024-11-01 13:27:27 +01:00
elnosh
fc9ed7c1fb include preimage and change in response from melt quote state check (#656) 2024-11-01 10:20:51 +01:00
Damian
b8dd43deaa Mint: settle mint-melt on same mint with different units externally (#651)
* WIP settle different units externally

* mint melt externally different units

* deprecated route return only sat

* comment

---------

Co-authored-by: callebtc <93376500+callebtc@users.noreply.github.com>
2024-10-30 21:59:57 +01:00
callebtc
09d007ec88 Refactor conditions and fix HTLC multisig (#643)
* refactor conditions and fix htlc multisig

* restore db/write.py

* safer check for P2PK secrets for SIG_ALL

* comment cleanup
2024-10-22 14:02:45 +02:00