lightningd: add listclosedchannels command.

Changelog-Added: JSON-RPC: `listclosedchannels` to show old, dead channels we previously had with peers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

doc/Makefile

@@ -52,6 +52,7 @@ MANPAGES := doc/lightning-cli.1 \
 	doc/lightning-invoicerequest.7 \
 	doc/lightning-keysend.7 \
 	doc/lightning-listchannels.7 \
+	doc/lightning-listclosedchannels.7 \
 	doc/lightning-listdatastore.7 \
 	doc/lightning-listforwards.7 \
 	doc/lightning-listfunds.7 \

doc/index.rst

@@ -78,6 +78,7 @@ Core Lightning Documentation
    lightning-invoicerequest <lightning-invoicerequest.7.md>
    lightning-keysend <lightning-keysend.7.md>
    lightning-listchannels <lightning-listchannels.7.md>
+   lightning-listclosedchannels <lightning-listclosedchannels.7.md>
    lightning-listconfigs <lightning-listconfigs.7.md>
    lightning-listdatastore <lightning-listdatastore.7.md>
    lightning-listforwards <lightning-listforwards.7.md>

doc/lightning-listclosedchannels.7.md

@@ -0,0 +1,79 @@
+lightning-listclosedchannels -- Get data on our closed historical channels
+==========================================================================
+
+SYNOPSIS
+--------
+
+**listclosedchannels** \[*id*\]
+
+DESCRIPTION
+-----------
+
+The **listclosedchannels** RPC command returns data on channels which
+are otherwise forgotten (more than 100 blocks after they're completely
+resolved onchain).
+
+If no *id* is supplied, then channel data on all historical channels are given.
+
+Supplying *id* will filter the results to only match channels to that peer.  Note that prior to v23.05, old peers were forgotten.
+
+RETURN VALUE
+------------
+
+[comment]: # (GENERATE-FROM-SCHEMA-START)
+On success, an object containing **closedchannels** is returned.  It is an array of objects, where each object contains:
+
+- **channel\_id** (hash): The full channel\_id (funding txid Xored with output number)
+- **opener** (string): Who initiated the channel (one of "local", "remote")
+- **private** (boolean): if False, we will not announce this channel
+- **total\_local\_commitments** (u64): Number of commitment transaction we made
+- **total\_remote\_commitments** (u64): Number of commitment transaction they made
+- **total\_htlcs\_sent** (u64): Number of HTLCs we ever sent
+- **funding\_txid** (txid): ID of the funding transaction
+- **funding\_outnum** (u32): The 0-based output number of the funding transaction which opens the channel
+- **leased** (boolean): Whether this channel was leased from `opener`
+- **total\_msat** (msat): total amount in the channel
+- **final\_to\_us\_msat** (msat): Our balance in final commitment transaction
+- **min\_to\_us\_msat** (msat): Least amount owed to us ever.  If the peer were to succesfully steal from us, this is the amount we would still retain.
+- **max\_to\_us\_msat** (msat): Most amount owed to us ever.  If we were to successfully steal from the peer, this is the amount we could potentially get.
+- **close\_cause** (string): What caused the channel to close (one of "unknown", "local", "user", "remote", "protocol", "onchain")
+- **peer\_id** (pubkey, optional): Peer public key (can be missing with pre-v23.05 closes!)
+- **short\_channel\_id** (short\_channel\_id, optional): The short\_channel\_id
+- **alias** (object, optional):
+  - **local** (short\_channel\_id, optional): An alias assigned by this node to this channel, used for outgoing payments
+  - **remote** (short\_channel\_id, optional): An alias assigned by the remote node to this channel, usable in routehints and invoices
+- **closer** (string, optional): Who initiated the channel close (only present if closing) (one of "local", "remote")
+- **channel\_type** (object, optional): channel\_type as negotiated with peer:
+  - **bits** (array of u32s): Each bit set in this channel\_type:
+    - Bit number
+  - **names** (array of strings): Feature name for each bit set in this channel\_type:
+    - Name of feature bit (one of "static\_remotekey/even", "anchor\_outputs/even", "anchors\_zero\_fee\_htlc\_tx/even", "scid\_alias/even", "zeroconf/even")
+- **funding\_fee\_paid\_msat** (msat, optional): How much we paid to lease the channel (iff `leased` is true and `opener` is local)
+- **funding\_fee\_rcvd\_msat** (msat, optional): How much they paid to lease the channel (iff `leased` is true and `opener` is remote)
+- **funding\_pushed\_msat** (msat, optional): How much `opener` pushed immediate (if non-zero)
+- **last\_commitment\_txid** (hash, optional): The final commitment tx's txid (or mutual close, if we accepted it).  Not present for some very old, small channels pre-0.7.0.
+- **last\_commitment\_fee\_msat** (msat, optional): The fee on `last_commitment_txid`
+
+[comment]: # (GENERATE-FROM-SCHEMA-END)
+
+On error the returned object will contain `code` and `message` properties,
+with `code` being one of the following:
+
+- -32602: If the given parameters are wrong.
+
+AUTHOR
+------
+
+Rusty Russell <<rusty@blockstream.com>>.
+
+SEE ALSO
+--------
+
+lightning-listpeerchannels(7)
+
+RESOURCES
+---------
+
+Main web site: <https://github.com/ElementsProject/lightning> Lightning
+
+[comment]: # ( SHA256STAMP:0c368cb41f46a2124e9b3f0b760494d1f4b9c3b248267f56b887fbf96f26e176)

doc/schemas/listclosedchannels.request.json

@@ -0,0 +1,13 @@
+{
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "type": "object",
+  "required": [],
+  "additionalProperties": false,
+  "added": "v23.05",
+  "properties": {
+    "id": {
+      "type": "pubkey",
+      "description": "If supplied, limits the channels to just the peer with the given ID, if it exists."
+    }
+  }
+}

doc/schemas/listclosedchannels.schema.json

@@ -0,0 +1,188 @@
+{
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "type": "object",
+  "additionalProperties": false,
+  "added": "v23.05",
+  "required": [
+    "closedchannels"
+  ],
+  "properties": {
+    "closedchannels": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "additionalProperties": true,
+        "required": [
+          "channel_id",
+          "opener",
+          "private",
+          "total_msat",
+          "total_local_commitments",
+          "total_remote_commitments",
+          "total_htlcs_sent",
+          "funding_txid",
+          "funding_outnum",
+          "leased",
+          "final_to_us_msat",
+          "min_to_us_msat",
+          "max_to_us_msat",
+          "close_cause"
+        ],
+        "properties": {
+          "peer_id": {
+            "type": "pubkey",
+            "description": "Peer public key (can be missing with pre-v23.05 closes!)"
+          },
+          "channel_id": {
+            "type": "hash",
+            "description": "The full channel_id (funding txid Xored with output number)"
+          },
+          "short_channel_id": {
+            "type": "short_channel_id",
+            "description": "The short_channel_id"
+          },
+          "alias": {
+            "type": "object",
+            "required": [],
+            "properties": {
+              "local": {
+                "type": "short_channel_id",
+                "description": "An alias assigned by this node to this channel, used for outgoing payments"
+              },
+              "remote": {
+                "type": "short_channel_id",
+                "description": "An alias assigned by the remote node to this channel, usable in routehints and invoices"
+              }
+            }
+          },
+          "opener": {
+            "type": "string",
+            "enum": [
+              "local",
+              "remote"
+            ],
+            "description": "Who initiated the channel"
+          },
+          "closer": {
+            "type": "string",
+            "enum": [
+              "local",
+              "remote"
+            ],
+            "description": "Who initiated the channel close (only present if closing)"
+          },
+          "private": {
+            "type": "boolean",
+            "description": "if False, we will not announce this channel"
+          },
+          "channel_type": {
+            "type": "object",
+            "description": "channel_type as negotiated with peer",
+            "additionalProperties": false,
+            "required": [
+              "bits",
+              "names"
+            ],
+            "properties": {
+              "bits": {
+                "type": "array",
+                "description": "Each bit set in this channel_type",
+                "items": {
+                  "type": "u32",
+                  "description": "Bit number"
+                }
+              },
+              "names": {
+                "type": "array",
+                "description": "Feature name for each bit set in this channel_type",
+                "items": {
+                  "type": "string",
+                  "enum": [
+                    "static_remotekey/even",
+                    "anchor_outputs/even",
+                    "anchors_zero_fee_htlc_tx/even",
+                    "scid_alias/even",
+                    "zeroconf/even"
+                  ],
+                  "description": "Name of feature bit"
+                }
+              }
+            }
+          },
+          "total_local_commitments": {
+            "type": "u64",
+            "description": "Number of commitment transaction we made"
+          },
+          "total_remote_commitments": {
+            "type": "u64",
+            "description": "Number of commitment transaction they made"
+          },
+          "total_htlcs_sent": {
+            "type": "u64",
+            "description": "Number of HTLCs we ever sent"
+          },
+          "funding_txid": {
+            "type": "txid",
+            "description": "ID of the funding transaction"
+          },
+          "funding_outnum": {
+            "type": "u32",
+            "description": "The 0-based output number of the funding transaction which opens the channel"
+          },
+          "leased": {
+            "type": "boolean",
+            "description": "Whether this channel was leased from `opener`"
+          },
+          "funding_fee_paid_msat": {
+            "type": "msat",
+            "description": "How much we paid to lease the channel (iff `leased` is true and `opener` is local)"
+          },
+          "funding_fee_rcvd_msat": {
+            "type": "msat",
+            "description": "How much they paid to lease the channel (iff `leased` is true and `opener` is remote)"
+          },
+          "funding_pushed_msat": {
+            "type": "msat",
+            "description": "How much `opener` pushed immediate (if non-zero)"
+          },
+          "total_msat": {
+            "type": "msat",
+            "description": "total amount in the channel"
+          },
+          "final_to_us_msat": {
+            "type": "msat",
+            "description": "Our balance in final commitment transaction"
+          },
+          "min_to_us_msat": {
+            "type": "msat",
+            "description": "Least amount owed to us ever.  If the peer were to succesfully steal from us, this is the amount we would still retain."
+          },
+          "max_to_us_msat": {
+            "type": "msat",
+            "description": "Most amount owed to us ever.  If we were to successfully steal from the peer, this is the amount we could potentially get."
+          },
+          "last_commitment_txid": {
+            "type": "hash",
+            "description": "The final commitment tx's txid (or mutual close, if we accepted it).  Not present for some very old, small channels pre-0.7.0."
+          },
+          "last_commitment_fee_msat": {
+            "type": "msat",
+            "description": "The fee on `last_commitment_txid`"
+          },
+          "close_cause": {
+            "type": "string",
+            "enum": [
+              "unknown",
+              "local",
+              "user",
+              "remote",
+              "protocol",
+              "onchain"
+            ],
+            "description": "What caused the channel to close"
+          }
+        }
+      }
+    }
+  }
+}

lightningd/Makefile

@@ -8,6 +8,7 @@ LIGHTNINGD_SRC :=				\
 	lightningd/closing_control.c		\
 	lightningd/coin_mvts.c			\
 	lightningd/dual_open_control.c		\
+	lightningd/closed_channel.c		\
 	lightningd/connect_control.c		\
 	lightningd/onion_message.c		\
 	lightningd/feerate.c			\
@@ -48,7 +49,6 @@ include wallet/Makefile
 
 LIGHTNINGD_HDRS :=				\
 	$(LIGHTNINGD_SRC:.c=.h)			\
-	lightningd/closed_channel.h		\
 	lightningd/channel_state.h		\
 	lightningd/channel_state_names_gen.h	\
 	$(WALLET_HDRS)

lightningd/closed_channel.c

@@ -0,0 +1,119 @@
+#include "config.h"
+#include <common/json_command.h>
+#include <common/json_param.h>
+#include <common/json_stream.h>
+#include <lightningd/channel.h>
+#include <lightningd/closed_channel.h>
+#include <lightningd/jsonrpc.h>
+#include <lightningd/lightningd.h>
+#include <lightningd/peer_control.h>
+#include <wallet/wallet.h>
+
+static void json_add_closed_channel(struct json_stream *response,
+				    const char *fieldname,
+				    const struct closed_channel *channel)
+{
+	json_object_start(response, fieldname);
+	if (channel->peer_id)
+		json_add_node_id(response, "peer_id", channel->peer_id);
+	json_add_channel_id(response, "channel_id", &channel->cid);
+	if (channel->scid)
+		json_add_short_channel_id(response, "short_channel_id",
+					  channel->scid);
+	if (channel->alias[LOCAL] || channel->alias[REMOTE]) {
+		json_object_start(response, "alias");
+		if (channel->alias[LOCAL])
+			json_add_short_channel_id(response, "local",
+						  channel->alias[LOCAL]);
+		if (channel->alias[REMOTE])
+			json_add_short_channel_id(response, "remote",
+						  channel->alias[REMOTE]);
+		json_object_end(response);
+	}
+	json_add_string(response, "opener",
+			channel->opener == LOCAL ? "local" : "remote");
+	if (channel->closer != NUM_SIDES)
+		json_add_string(response, "closer", channel->closer == LOCAL ?
+						    "local" : "remote");
+
+	json_add_bool(response, "private",
+		      !(channel->channel_flags & CHANNEL_FLAGS_ANNOUNCE_CHANNEL));
+
+	json_add_channel_type(response, "channel_type", channel->type);
+	json_add_u64(response, "total_local_commitments",
+		     channel->next_index[LOCAL] - 1);
+	json_add_u64(response, "total_remote_commitments",
+		     channel->next_index[REMOTE] - 1);
+	json_add_u64(response, "total_htlcs_sent", channel->next_htlc_id);
+	json_add_txid(response, "funding_txid", &channel->funding.txid);
+	json_add_num(response, "funding_outnum", channel->funding.n);
+	json_add_bool(response, "leased", channel->leased);
+	if (channel->leased) {
+		if (channel->opener == LOCAL)
+			json_add_amount_msat(response, "funding_fee_paid_msat",
+					     channel->push);
+		else
+			json_add_amount_msat(response, "funding_fee_rcvd_msat",
+					     channel->push);
+	} else if (!amount_msat_eq(channel->push, AMOUNT_MSAT(0)))
+		json_add_amount_msat(response, "funding_pushed_msat",
+				     channel->push);
+
+	json_add_amount_sat_msat(response, "total_msat", channel->funding_sats);
+	json_add_amount_msat(response, "final_to_us_msat", channel->our_msat);
+	json_add_amount_msat(response, "min_to_us_msat",
+			     channel->msat_to_us_min);
+	json_add_amount_msat(response, "max_to_us_msat",
+			     channel->msat_to_us_max);
+	if (channel->last_tx && !invalid_last_tx(channel->last_tx)) {
+		struct bitcoin_txid txid;
+		bitcoin_txid(channel->last_tx, &txid);
+
+		json_add_txid(response, "last_commitment_txid", &txid);
+		json_add_amount_sat_msat(response, "last_commitment_fee_msat",
+					 bitcoin_tx_compute_fee(channel->last_tx));
+	}
+	json_add_string(response, "close_cause",
+			channel_change_state_reason_str(channel->state_change_cause));
+	json_object_end(response);
+}
+
+static struct command_result *json_listclosedchannels(struct command *cmd,
+						      const char *buffer,
+						      const jsmntok_t *obj UNNEEDED,
+						      const jsmntok_t *params)
+{
+	struct node_id *peer_id;
+	struct json_stream *response;
+	struct closed_channel **chans;
+
+	if (!param(cmd, buffer, params,
+		   p_opt("id", param_node_id, &peer_id),
+		   NULL))
+		return command_param_failed();
+
+	response = json_stream_success(cmd);
+	json_array_start(response, "closedchannels");
+
+	chans = wallet_load_closed_channels(cmd, cmd->ld->wallet);
+	for (size_t i = 0; i < tal_count(chans); i++) {
+		if (peer_id) {
+			if (!chans[i]->peer_id)
+				continue;
+			if (!node_id_eq(chans[i]->peer_id, peer_id))
+				continue;
+		}
+		json_add_closed_channel(response, NULL, chans[i]);
+	}
+	json_array_end(response);
+
+	return command_success(cmd, response);
+}
+
+static const struct json_command listclosedchannels_command = {
+	"listclosedchannels",
+	"network",
+	json_listclosedchannels,
+	"Show historical (dead) channels."
+};
+AUTODATA(json_command, &listclosedchannels_command);

lightningd/closed_channel.h

@@ -2,6 +2,10 @@
 #ifndef LIGHTNING_LIGHTNINGD_CLOSED_CHANNEL_H
 #define LIGHTNING_LIGHTNINGD_CLOSED_CHANNEL_H
 #include "config.h"
+#include <bitcoin/tx.h>
+#include <common/channel_id.h>
+#include <common/htlc.h>
+#include <lightningd/channel_state.h>
 
 struct closed_channel {
 	/* This is often deleted on older nodes! */

tests/test_closing.py

@@ -95,14 +95,50 @@ def test_closing_simple(node_factory, bitcoind, chainparams):
         'ONCHAIN:All outputs resolved: waiting 90 more blocks before forgetting channel'
     ])
 
+    # Capture both side's image of channel before it's dead.
+    l1channel = only_one(l1.rpc.listpeerchannels(l2.info['id'])['channels'])
+    l2channel = only_one(l2.rpc.listpeerchannels(l1.info['id'])['channels'])
+
     # Make sure both have forgotten about it
     bitcoind.generate_block(90)
-    wait_for(lambda: len(l1.rpc.listchannels()['channels']) == 0)
+    wait_for(lambda: len(l1.rpc.listpeerchannels()['channels']) == 0)
-    wait_for(lambda: len(l2.rpc.listchannels()['channels']) == 0)
+    wait_for(lambda: len(l2.rpc.listpeerchannels()['channels']) == 0)
 
     # The entry in the channels table should still be there
     assert l1.db_query("SELECT count(*) as c FROM channels;")[0]['c'] == 1
     assert l2.db_query("SELECT count(*) as c FROM channels;")[0]['c'] == 1
+    assert l1.db_query("SELECT count(*) as p FROM peers;")[0]['p'] == 1
+    assert l2.db_query("SELECT count(*) as p FROM peers;")[0]['p'] == 1
+
+    # Test listclosedchannels is correct.
+    l1closedchannel = only_one(l1.rpc.listclosedchannels()['closedchannels'])
+    l2closedchannel = only_one(l2.rpc.listclosedchannels(l1.info['id'])['closedchannels'])
+
+    # These fields do not appear in listpeerchannels!
+    l1_only_closed = {'total_local_commitments': 2,
+                      'total_remote_commitments': 2,
+                      'total_htlcs_sent': 1,
+                      'leased': False,
+                      'close_cause': 'user'}
+    l2_only_closed = {'total_local_commitments': 2,
+                      'total_remote_commitments': 2,
+                      'total_htlcs_sent': 0,
+                      'leased': False,
+                      'close_cause': 'remote'}
+
+    # These fields have different names
+    renamed = {'last_commitment_txid': 'scratch_txid',
+               'last_commitment_fee_msat': 'last_tx_fee_msat',
+               'final_to_us_msat': 'to_us_msat'}
+
+    for chan, closedchan, onlyclosed in (l1channel, l1closedchannel, l1_only_closed), (l2channel, l2closedchannel, l2_only_closed):
+        for k, v in closedchan.items():
+            if k in renamed:
+                assert chan[renamed[k]] == v
+            elif k in onlyclosed:
+                assert closedchan[k] == onlyclosed[k]
+            else:
+                assert chan[k] == v
 
     assert account_balance(l1, channel_id) == 0
     assert account_balance(l2, channel_id) == 0