fetchinvoice: try to connect to note if we can't find a path for messages.

This also adds a `fetchinvoice-noconnect` option to suppress it too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: `fetchinvoice` and `sendinvoice` will connect directly if they can't find an onionmessage route.
Fixes: #4624

doc/lightning-fetchinvoice.7

@@ -15,6 +15,11 @@ an actual invoice that can be paid\.  It highlights any changes between the
 offer and the returned invoice\.
 
 
+If \fBfetchinvoice-noconnect\fR is not specified in the configuation, it
+will connect to the destination in the (currently common!) case where it
+cannot find a route which supports \fBoption_onion_messages\fR\.
+
+
 The offer must not contain \fIsend_invoice\fR; see \fBlightning-sendinvoice\fR(7)\.
 
 
@@ -114,4 +119,4 @@ Rusty Russell \fI<rusty@rustcorp.com.au\fR> is mainly responsible\.
 
 Main web site: \fIhttps://github.com/ElementsProject/lightning\fR
 
-\" SHA256STAMP:532248cb5adbadb10367fdbddc2da7af0eeac50b29709abec2e1e8b178197b7c
+\" SHA256STAMP:8343ee7fe4d8413760a47a9d2657c4557734fa67af5bfec582daf780828ca675

doc/lightning-fetchinvoice.7.md

@@ -15,6 +15,10 @@ The **fetchinvoice** RPC command contacts the issuer of an *offer* to get
 an actual invoice that can be paid.  It highlights any changes between the
 offer and the returned invoice.
 
+If **fetchinvoice-noconnect** is not specified in the configuation, it
+will connect to the destination in the (currently common!) case where it
+cannot find a route which supports `option_onion_messages`.
+
 The offer must not contain *send_invoice*; see lightning-sendinvoice(7).
 
 *msatoshi* is required if the *offer* does not specify

doc/lightning-offerout.7

@@ -100,10 +100,10 @@ Rusty Russell \fI<rusty@rustcorp.com.au\fR> is mainly responsible\.
 
 .SH SEE ALSO
 
-\fBlightning-offer\fR(7), \fBlightning-listoffers\fR(7), \fBlightning-disableoffer\fR(7)\.
+\fBlightning-sendinvoice\fR(7), \fBlightning-offer\fR(7), \fBlightning-listoffers\fR(7), \fBlightning-disableoffer\fR(7)\.
 
 .SH RESOURCES
 
 Main web site: \fIhttps://github.com/ElementsProject/lightning\fR
 
-\" SHA256STAMP:ccf9c53e1189ef9138954beed8fe5e5318e2dfebb53fde2ee20a8777aff255b5
+\" SHA256STAMP:823219aff5dc06ab3b810442048b6cf733210c3eae80567327dc396e5f7987c8

doc/lightning-offerout.7.md

@@ -85,7 +85,7 @@ Rusty Russell <<rusty@rustcorp.com.au>> is mainly responsible.
 SEE ALSO
 --------
 
-lightning-offer(7), lightning-listoffers(7), lightning-disableoffer(7).
+lightning-sendinvoice(7), lightning-offer(7), lightning-listoffers(7), lightning-disableoffer(7).
 
 RESOURCES
 ---------

doc/lightning-sendinvoice.7

@@ -6,7 +6,7 @@ lightning-sendinvoice - Command for send an invoice for an offer
 \fB(WARNING: experimental-offers only)\fR
 
 
-\fBsendinvoice\fR \fIoffer\fR [\fIlabel\fR] [\fImsatoshi\fR] [\fItimeout\fR] [\fIquantity\fR]
+\fBsendinvoice\fR \fIoffer\fR \fIlabel\fR [\fImsatoshi\fR] [\fItimeout\fR] [\fIquantity\fR]
 
 .SH DESCRIPTION
 
@@ -15,6 +15,11 @@ issuer of an \fIoffer\fR for it to pay: the offer must contain
 \fIsend_invoice\fR; see \fBlightning-fetchinvoice\fR(7)\.
 
 
+If \fBfetchinvoice-noconnect\fR is not specified in the configuation, it
+will connect to the destination in the (currently common!) case where it
+cannot find a route which supports \fBoption_onion_messages\fR\.
+
+
 \fIoffer\fR is the bolt12 offer string beginning with "lno1"\.
 
 
@@ -98,4 +103,4 @@ Rusty Russell \fI<rusty@rustcorp.com.au\fR> is mainly responsible\.
 
 Main web site: \fIhttps://github.com/ElementsProject/lightning\fR
 
-\" SHA256STAMP:de314ada333bec6eb2bad2ad1410201c8c99c492203cf178dfacd95d7b74c0f9
+\" SHA256STAMP:c01a52cc2ab1f5badf212481e3aefb91c8a4c17df93d86bd5f98588f2679d8d6

doc/lightning-sendinvoice.7.md

@@ -6,7 +6,7 @@ SYNOPSIS
 
 **(WARNING: experimental-offers only)**
 
-**sendinvoice** *offer* \[*label*\] \[*msatoshi*\] \[*timeout*\] \[*quantity*\]
+**sendinvoice** *offer* *label* \[*msatoshi*\] \[*timeout*\] \[*quantity*\]
 
 DESCRIPTION
 -----------
@@ -15,6 +15,10 @@ The **sendinvoice** RPC command creates and sends an invoice to the
 issuer of an *offer* for it to pay: the offer must contain
 *send_invoice*; see lightning-fetchinvoice(7).
 
+If **fetchinvoice-noconnect** is not specified in the configuation, it
+will connect to the destination in the (currently common!) case where it
+cannot find a route which supports `option_onion_messages`.
+
 *offer* is the bolt12 offer string beginning with "lno1".
 
 *label* is the unique label to use for this invoice.

doc/lightningd-config.5

@@ -145,7 +145,6 @@ What log level to print out: options are io, debug, info, unusual,
 broken\.  If \fISUBSYSTEM\fR is supplied, this sets the logging level
 for any subsystem containing that string\.  Subsystems include:
 
-
 .RS
 .IP \[bu]
 \fIlightningd\fR: The main lightning daemon
@@ -171,7 +170,6 @@ for any subsystem containing that string\.  Subsystems include:
   The following subsystems exist for each channel, where N is an incrementing
 internal integer id assigned for the lifetime of the channel:
 
-
 .RS
 .IP \[bu]
 \fIopeningd-chan#N\fR: Each opening / idling daemon
@@ -587,6 +585,13 @@ This usually requires \fBexperimental-onion-messages\fR as well\.  See
 \fBlightning-offer\fR(7) and \fBlightning-fetchinvoice\fR(7)\.
 
 
+ \fBfetchinvoice-noconnect\fR
+
+
+Specifying this prevents \fBfetchinvoice\fR and \fBsendinvoice\fR from
+trying to connect directly to the offering node as a last resort\.
+
+
  \fBexperimental-shutdown-wrong-funding\fR
 
 
@@ -630,4 +635,4 @@ Main web site: \fIhttps://github.com/ElementsProject/lightning\fR
 Note: the modules in the ccan/ directory have their own licenses, but
 the rest of the code is covered by the BSD-style MIT license\.
 
-\" SHA256STAMP:55425fe062d1f3365ada296e11e57ede0bdda345b1a70583dd82de2da6e55988
+\" SHA256STAMP:40c9f5e9e4ee5257e25a1fc196d2c85c3bc5b21d3f390a4e7fafa031c4e7ad5e

doc/lightningd-config.5.md

@@ -484,6 +484,11 @@ corresponding functionality, which are in draft status as BOLT12.
 This usually requires **experimental-onion-messages** as well.  See
 lightning-offer(7) and lightning-fetchinvoice(7).
 
+ **fetchinvoice-noconnect**
+
+Specifying this prevents `fetchinvoice` and `sendinvoice` from
+trying to connect directly to the offering node as a last resort.
+
  **experimental-shutdown-wrong-funding**
 
 Specifying this allows the `wrong_funding` field in shutdown: if a

plugins/fetchinvoice.c

@@ -26,6 +26,7 @@
 
 static struct gossmap *global_gossmap;
 static struct node_id local_id;
+static bool disable_connect = false;
 static LIST_HEAD(sent_list);
 
 struct sent {
@@ -37,6 +38,8 @@ struct sent {
 	struct command *cmd;
 	/* The offer we are trying to get an invoice/payment for. */
 	struct tlv_offer *offer;
+	/* Path to use. */
+	struct node_id *path;
 
 	/* The invreq we sent, OR the invoice we sent */
 	struct tlv_invoice_request *invreq;
@@ -516,6 +519,64 @@ static bool can_carry_onionmsg(const struct gossmap *map,
 	return n && gossmap_node_get_feature(map, n, OPT_ONION_MESSAGES) != -1;
 }
 
+/* Create path to node which can carry onion messages; if it can't find
+ * one, create singleton path and sets @try_connect.  */
+static struct node_id *path_to_node(const tal_t *ctx,
+				    struct gossmap *gossmap,
+				    const struct pubkey32 *node32_id,
+				    bool *try_connect)
+{
+	const struct gossmap_node *dst;
+	struct node_id *nodes, dstid;
+
+	/* FIXME: Use blinded path if avail. */
+	gossmap_guess_node_id(gossmap, node32_id, &dstid);
+	dst = gossmap_find_node(gossmap, &dstid);
+	if (!dst) {
+		nodes = tal_arr(ctx, struct node_id, 1);
+		/* We don't know the pubkey y-sign, but sendonionmessage will
+		 * fix it up if we guess wrong. */
+		nodes[0].k[0] = SECP256K1_TAG_PUBKEY_EVEN;
+		secp256k1_xonly_pubkey_serialize(secp256k1_ctx,
+						 nodes[0].k+1,
+						 &node32_id->pubkey);
+		/* Since it's not it gossmap, we don't know how to connect,
+		 * so don't try. */
+		*try_connect = false;
+		return nodes;
+	} else {
+		struct route_hop *r;
+		const struct dijkstra *dij;
+		const struct gossmap_node *src;
+
+		/* If we don't exist in gossip, routing can't happen. */
+		src = gossmap_find_node(gossmap, &local_id);
+		if (!src)
+			goto go_direct_dst;
+
+		dij = dijkstra(tmpctx, gossmap, dst, AMOUNT_MSAT(0), 0,
+			       can_carry_onionmsg, route_score_shorter, NULL);
+
+		r = route_from_dijkstra(tmpctx, gossmap, dij, src, AMOUNT_MSAT(0), 0);
+		if (!r)
+			goto go_direct_dst;
+
+		*try_connect = false;
+		nodes = tal_arr(ctx, struct node_id, tal_count(r));
+		for (size_t i = 0; i < tal_count(r); i++)
+			nodes[i] = r[i].node_id;
+		return nodes;
+	}
+
+go_direct_dst:
+	/* Try direct route, maybe it's connected? */
+	nodes = tal_arr(ctx, struct node_id, 1);
+	gossmap_node_get_id(gossmap, dst, &nodes[0]);
+	*try_connect = true;
+	return nodes;
+}
+
+/* Send this message down this path, with blinded reply path */
 static struct command_result *send_message(struct command *cmd,
 					   struct sent *sent,
 					   const char *msgfield,
@@ -526,70 +587,25 @@ static struct command_result *send_message(struct command *cmd,
 					    const jsmntok_t *result UNUSED,
 					    struct sent *sent))
 {
-	const struct gossmap_node *dst;
-	struct gossmap *gossmap = get_gossmap(cmd->plugin);
 	struct pubkey *backwards;
 	struct onionmsg_path **path;
 	struct pubkey blinding;
 	struct out_req *req;
-	struct node_id dstid, *nodes;
-
-	/* FIXME: Use blinded path if avail. */
-	gossmap_guess_node_id(gossmap, sent->offer->node_id, &dstid);
-	dst = gossmap_find_node(gossmap, &dstid);
-	if (!dst) {
-		/* Try direct. */
-		struct pubkey *us = tal_arr(tmpctx, struct pubkey, 1);
-		if (!pubkey_from_node_id(&us[0], &local_id))
-			abort();
-		backwards = us;
-
-		nodes = tal_arr(tmpctx, struct node_id, 1);
-		/* We don't know the pubkey y-sign, but sendonionmessage will
-		 * fix it up if we guess wrong. */
-		nodes[0].k[0] = SECP256K1_TAG_PUBKEY_EVEN;
-		secp256k1_xonly_pubkey_serialize(secp256k1_ctx,
-						 nodes[0].k+1,
-						 &sent->offer->node_id->pubkey);
-	} else {
-		struct route_hop *r;
-		const struct dijkstra *dij;
-		const struct gossmap_node *src;
-
-		/* If we don't exist in gossip, routing can't happen. */
-		src = gossmap_find_node(gossmap, &local_id);
-		if (!src)
-			return command_fail(cmd, PAY_ROUTE_NOT_FOUND,
-					    "We don't have any channels");
-
-		dij = dijkstra(tmpctx, gossmap, dst, AMOUNT_MSAT(0), 0,
-			       can_carry_onionmsg, route_score_shorter, NULL);
-
-		r = route_from_dijkstra(tmpctx, gossmap, dij, src, AMOUNT_MSAT(0), 0);
-		if (!r)
-			/* FIXME: try connecting directly. */
-			return command_fail(cmd, OFFER_ROUTE_NOT_FOUND,
-					    "Can't find route");
 
 	/* FIXME: Maybe we should allow this? */
-		if (tal_bytelen(r) == 0)
+	if (tal_bytelen(sent->path) == 0)
 		return command_fail(cmd, PAY_ROUTE_NOT_FOUND,
 				    "Refusing to talk to ourselves");
 
-		nodes = tal_arr(tmpctx, struct node_id, tal_count(r));
-		for (size_t i = 0; i < tal_count(r); i++)
-			nodes[i] = r[i].node_id;
-
 	/* Reverse path is offset by one: we are the final node. */
-		backwards = tal_arr(tmpctx, struct pubkey, tal_count(r));
+	backwards = tal_arr(tmpctx, struct pubkey, tal_count(sent->path));
-		for (size_t i = 0; i < tal_count(r) - 1; i++) {
+	for (size_t i = 0; i < tal_count(sent->path) - 1; i++) {
-			if (!pubkey_from_node_id(&backwards[tal_count(r)-2-i],
+		if (!pubkey_from_node_id(&backwards[tal_count(sent->path)-2-i],
-						 &nodes[i]))
+					 &sent->path[i]))
 			abort();
 	}
-		if (!pubkey_from_node_id(&backwards[tal_count(r)-1], &local_id))
+	if (!pubkey_from_node_id(&backwards[tal_count(sent->path)-1], &local_id))
 		abort();
-	}
 
 	/* Ok, now make reply for onion_message */
 	path = make_blindedpath(tmpctx, backwards, &blinding,
@@ -600,10 +616,10 @@ static struct command_result *send_message(struct command *cmd,
 				    forward_error,
 				    sent);
 	json_array_start(req->js, "hops");
-	for (size_t i = 0; i < tal_count(nodes); i++) {
+	for (size_t i = 0; i < tal_count(sent->path); i++) {
 		json_object_start(req->js, NULL);
-		json_add_node_id(req->js, "id", &nodes[i]);
+		json_add_node_id(req->js, "id", &sent->path[i]);
-		if (i == tal_count(nodes) - 1)
+		if (i == tal_count(sent->path) - 1)
 			json_add_hex_talarr(req->js, msgfield, msgval);
 		json_object_end(req->js);
 	}
@@ -650,6 +666,52 @@ static struct command_result *prepare_inv_timeout(struct command *cmd,
 	return sendonionmsg_done(cmd, buf, result, sent);
 }
 
+/* We've connected (if we tried), so send the invreq. */
+static struct command_result *
+sendinvreq_after_connect(struct command *cmd,
+			 const char *buf UNUSED,
+			 const jsmntok_t *result UNUSED,
+			 struct sent *sent)
+{
+	u8 *rawinvreq = tal_arr(tmpctx, u8, 0);
+	towire_invoice_request(&rawinvreq, sent->invreq);
+
+	return send_message(cmd, sent, "invoice_request", rawinvreq,
+			    sendonionmsg_done);
+}
+
+/* We can't find a route, so we're going to try to connect, then just blast it
+ * to them. */
+static struct command_result *
+connect_direct(struct command *cmd,
+	       const struct node_id *dst,
+	       struct command_result *(*cb)(struct command *command,
+					    const char *buf,
+					    const jsmntok_t *result,
+					    struct sent *sent),
+	       struct sent *sent)
+{
+	struct out_req *req;
+
+	if (disable_connect) {
+		plugin_notify_message(cmd, LOG_UNUSUAL,
+				      "Cannot find route, but"
+				      " fetchplugin-noconnect set:"
+				      " trying direct anyway to %s",
+				      type_to_string(tmpctx, struct node_id,
+						     dst));
+		return cb(cmd, NULL, NULL, sent);
+	}
+
+	plugin_notify_message(cmd, LOG_INFORM,
+			      "Cannot find route, trying connect to %s directly",
+			      type_to_string(tmpctx, struct node_id, dst));
+
+	req = jsonrpc_request_start(cmd->plugin, cmd, "connect", cb, cb, sent);
+	json_add_node_id(req->js, "id", dst);
+	return send_outreq(cmd->plugin, req);
+}
+
 static struct command_result *invreq_done(struct command *cmd,
 					  const char *buf,
 					  const jsmntok_t *result,
@@ -657,7 +719,7 @@ static struct command_result *invreq_done(struct command *cmd,
 {
 	const jsmntok_t *t;
 	char *fail;
-	u8 *rawinvreq;
+	bool try_connect;
 
 	/* Get invoice request */
 	t = json_get_member(buf, result, "bolt12");
@@ -750,10 +812,14 @@ static struct command_result *invreq_done(struct command *cmd,
 		}
 	}
 
-	rawinvreq = tal_arr(tmpctx, u8, 0);
+	sent->path = path_to_node(sent, get_gossmap(cmd->plugin),
-	towire_invoice_request(&rawinvreq, sent->invreq);
+				  sent->offer->node_id,
-	return send_message(cmd, sent, "invoice_request", rawinvreq,
+				  &try_connect);
-			    sendonionmsg_done);
+	if (try_connect)
+		return connect_direct(cmd, &sent->path[0],
+				      sendinvreq_after_connect, sent);
+
+	return sendinvreq_after_connect(cmd, NULL, NULL, sent);
 }
 
 /* Fetches an invoice for this offer, and makes sure it corresponds. */
@@ -987,6 +1053,18 @@ static struct command_result *invoice_payment(struct command *cmd,
 	return command_hook_success(cmd);
 }
 
+/* We've connected (if we tried), so send the invoice. */
+static struct command_result *
+sendinvoice_after_connect(struct command *cmd,
+			  const char *buf UNUSED,
+			  const jsmntok_t *result UNUSED,
+			  struct sent *sent)
+{
+	u8 *rawinv = tal_arr(tmpctx, u8, 0);
+	towire_invoice(&rawinv, sent->inv);
+	return send_message(cmd, sent, "invoice", rawinv, prepare_inv_timeout);
+}
+
 static struct command_result *createinvoice_done(struct command *cmd,
 						 const char *buf,
 						 const jsmntok_t *result,
@@ -994,7 +1072,7 @@ static struct command_result *createinvoice_done(struct command *cmd,
 {
 	const jsmntok_t *invtok = json_get_member(buf, result, "bolt12");
 	char *fail;
-	u8 *rawinv;
+	bool try_connect;
 
 	/* Replace invoice with signed one */
 	tal_free(sent->inv);
@@ -1014,9 +1092,14 @@ static struct command_result *createinvoice_done(struct command *cmd,
 				    "Bad createinvoice response %s", fail);
 	}
 
-	rawinv = tal_arr(tmpctx, u8, 0);
+	sent->path = path_to_node(sent, get_gossmap(cmd->plugin),
-	towire_invoice(&rawinv, sent->inv);
+				  sent->offer->node_id,
-	return send_message(cmd, sent, "invoice", rawinv, prepare_inv_timeout);
+				  &try_connect);
+	if (try_connect)
+		return connect_direct(cmd, &sent->path[0],
+				      sendinvoice_after_connect, sent);
+
+	return sendinvoice_after_connect(cmd, NULL, NULL, sent);
 }
 
 static struct command_result *sign_invoice(struct command *cmd,
@@ -1377,6 +1460,8 @@ int main(int argc, char *argv[])
 	            NULL, 0,
 		    hooks, ARRAY_SIZE(hooks),
 		    NULL, 0,
-		    /* No options */
+		    plugin_option("fetchinvoice-noconnect", "flag",
+				  "Don't try to connect directly to fetch an invoice.",
+				  flag_option, &disable_connect),
 		    NULL);
 }