374 Commits

Author SHA1 Message Date
Olaoluwa Osuntokun
5a8255550b discovery: revamp recent rejects map
Similar to the prior commit, in this commit, we move to using a basic
LRU cache to store the set of prior rejected messages.
2021-11-04 15:21:20 -07:00
Olaoluwa Osuntokun
8627b5d128 discovery: revamp premature update map
Turns out we need it right now to handle some low latency race
conditions in our integration tests, so we'll opt to simply cap the size
of it to a low amount. We use a basic LRU caching mechainsm.

Fixes https://github.com/lightningnetwork/lnd/issues/5076
2021-11-04 15:21:18 -07:00
Oliver Gugger
e79d59dd4c multi: use key locator for lnwallet.MessageSigner
To simplify the message signing API even further, we refactor the
lnwallet.MessageSigner interface to use a key locator instead of the
public key to identify which key should be signed with.
2021-10-08 12:06:52 +02:00
Andras Banki-Horvath
11cf4216e4 multi: move all channelstate operations to ChannelStateDB 2021-09-29 17:00:03 +02:00
yyforyongyu
3204e2d74b multi: add shutdown logs in subservers
This commit adds a simple shutdown to every subserver to assist
debugging.
2021-09-15 19:52:03 +08:00
Joost Jager
3f775778c3 channeldb+routing: add tx parameter
Adds an optional tx parameter to ForAllOutgoingChannels and FetchChannel
so that data can be queried within the context of an existing database
transaction.
2021-08-24 13:43:24 +02:00
Wilmer Paulino
186237ca73 discovery: demote err log to debug from processZombieUpdate
This log can be "spammy" while nodes throughout the network have yet to
upgrade to v0.13.0-beta, which includes several enhancements to prevent
the broadcast of zombie edges/updates.
2021-05-26 13:33:41 -07:00
Andras Banki-Horvath
14c851c8fc kvdb: move channeldb/kvdb to top level 2021-05-07 14:18:56 +02:00
Wilmer Paulino
0ada2288cc discovery: prevent logging certain validation barrier errors 2021-04-29 15:53:07 -07:00
Olaoluwa Osuntokun
5d1574f566 Merge pull request #2522 from roeierez/cleanup_server_error
Cleanup in case of server failed to start
2021-04-22 13:00:22 -07:00
Olaoluwa Osuntokun
a9f1b341be discovery: update zombie resurrection test w/ new logic
In this commit, we update the existing zombie resurrection test to
ensure that if we prune an edge and another pubkey is marked as nil,
that we only accept a resurrection channel update from the node the we
originally pruned if the pruning decision was one sided.
2021-04-21 13:56:35 -05:00
Conner Fromknecht
f28a98aa6f discovery/gossiper: only parse zombie pubkey if non-empty 2021-04-21 13:56:16 -05:00
Conner Fromknecht
4baee9537b discovery/gossiper: move zombie handling to helper 2021-04-21 13:56:11 -05:00
Olaoluwa Osuntokun
bb6aca1130 discovery: add unit tests for chan anns in the reject cache 2021-04-14 15:19:27 -07:00
Olaoluwa Osuntokun
c959ecc4c9 discovery: always add chan announcements to the reject cache if err != ErrIgnored
In this commit, we make a change to always add chan announcements to the
reject cache if we didn't reject them for already existing.  Without
this change, if we end up rejecting a channel announcement say because
the channel is already spent or the funding transaction doesn't exist,
then we'll end up continually re-validating the same set of channels we
know will fail, when they're sent to us by peers.

Fixes #5191
2021-04-14 15:19:14 -07:00
Roei Erez
3223df74e5 channelnotifier+discover+invoices: return error in Stop functions
In order to be consistent with other sub systems an error is now
returned from the Stop functions.
This also allows writing a generic cleanup mechanism to stop all
sub systems in case of a failure.
2021-04-13 13:26:01 +03:00
Wilmer Paulino
393111cea9 discovery+routing: cancel dependent jobs if parent validation fails
Previously, we would always allow dependent jobs to be processed,
regardless of the result of its parent job's validation. This isn't
correct, as a parent job contains actions necessary to successfully
process a dependent job. A prime example of this can be found within the
AuthenticatedGossiper, where an incoming channel announcement and update
are both processed, but if the channel announcement job fails to
complete, then the gossiper is unable to properly validate the update.
This commit aims to address this by preventing the dependent jobs to
run.
2021-03-23 11:56:51 -07:00
Wilmer Paulino
e713205eea discovery: add missing error channel sends in processNetworkAnnouncement
Without the error channel sends, we would block the gossip message
stream upon receiving a premature channel announcement.
2021-03-23 11:56:31 -07:00
Olaoluwa Osuntokun
9a6bb19770 lnwire: prep wire messages for TLV extensions
Messages:
- UpdateFulfillHTLC
- UpdateFee
- UpdateFailMalformedHTLC
- UpdateFailHTLC
- UpdateAddHTLC
- Shutdown
- RevokeAndAck
- ReplyShortChanIDsEnd
- ReplyChannelRange
- QueryShortChanIDs
- QueryChannelRange
- NodeAnnouncement
- Init
- GossipTimestampRange
- FundingSigned
- FundingLocked
- FundingCreated
- CommitSig
- ClosingSigned
- ChannelUpdate
- ChannelReestablish
- ChannelAnnouncement
- AnnounceSignatures

lnwire: update quickcheck tests, use constant for Error

multi: update unit tests to pass deep equal assertions with messages

In this commit, we update a series of unit tests in the code base to now
pass due to the new wire message encode/decode logic. In many instances,
we'll now manually set the extra bytes to an empty byte slice to avoid
comparisons that fail due to one message having an empty byte slice and
the other having a nil pointer.
2021-02-24 17:31:55 +01:00
Olaoluwa Osuntokun
dd6f0ba931 discovery+lnwire: remove embedding within ReplyChannelRange
In order to prep for allowing TLV extensions for the `ReplyChannelRange`
and `QueryChannelRange` messages, we'll need to remove the struct
embedding as is. If we don't remove this, then we'll attempt to decode
TLV extensions from both the embedded and outer struct.

All relevant call sites have been updated to reflect this minor change.
2021-02-24 17:31:55 +01:00
Johan T. Halseth
926005aefa discovery/gossiper: ignore remote ChannelAnnouncement for own channel
To avoid learning about our own channel we have already closed and
removed from our graph, we ignore all ChannelAnns for channels we are
involved in.
2021-02-11 15:13:51 +01:00
Johan T. Halseth
e8f7a11470 gossiper_test: split keys into self/remote
To make it more clear what is local and remote messages, we change to
use `selfKey` only for local messages.
2021-02-11 15:13:51 +01:00
Johan T. Halseth
4268bcc9f9 gossiper_test: combine local/remote chan ann
It will be the same announcement, no need to distinguish.
2021-02-11 15:13:51 +01:00
Johan T. Halseth
f047c3517f discovery/gossiper: remove source pubkey from ProcessLocalAnnouncement params 2021-02-11 15:13:51 +01:00
Olaoluwa Osuntokun
355b6a260f Merge pull request #4964 from halseth/gossip-local-no-batch
channeldb+routing+gossiper: add local updates to graph immediately
2021-02-10 18:00:47 -08:00
Conner Fromknecht
1ee5eb97d5 Merge pull request #5006 from wpaulino/new-rate-limit-chan-updates
discovery: use token bucket based rate limiting to throttle gossip
2021-02-10 17:31:41 -08:00
Conner Fromknecht
5afc6b9284 Merge pull request #4945 from cfromknecht/no-graph-sync
discovery: no graph sync
2021-02-10 17:07:24 -08:00
Wilmer Paulino
83a0d03c0b discovery: use token bucket based rate limiting to throttle gossip
The recently added gossip throttling was shown to be too aggressive,
especially with our auto channel enable/disable signaling. We switch to
a token bucket based system instead as it's based on time, rather than a
block height which isn't constantly updated at a given rate.
2021-02-10 16:21:13 -08:00
Wilmer Paulino
bfc8523873 Revert "discovery: add new option to toggle gossip rate limiting"
This reverts commit 13a2598ded.
2021-02-10 15:45:48 -08:00
Johan T. Halseth
c9afc93151 discovery/gossiper: add local updates to graph immediately
Since the batch interval can potentially be long, adding local updates
to the graph could be slow. This would slow down operations like adding
our own channel update and announcements during the funding process, and
updating edge policies for local channels.

Now we instead check whether the update is remote or not, and only for
remote updates use the SchedulerOption to lazily add them to the graph.
2021-02-10 23:54:03 +01:00
Johan T. Halseth
7e34132c53 routing: let graph methods take scheduler option 2021-02-10 23:54:03 +01:00
Wilmer Paulino
904003fbcb discovery: use source of ann upon confirmed channel ann batch
We do this instead of using the source of the AnnounceSignatures
message, as we filter out the source when broadcasting any
announcements, leading to the remote node not receiving our channel
update. Note that this is done more for the sake of correctness and to
address a flake within the integration tests, as channel updates are
sent directly and reliably to channel counterparts.
2021-02-10 13:22:28 -08:00
Conner Fromknecht
58e924ad1c discovery: don't historical sync when NumActiveSyncers == 0
Currently when numgraphsyncpeers=0, lnd will still attempt to perform
an initial historical sync. We change this behavior here to forgoe
historical sync entirely when numgraphsyncpeers is zero, since the
routing table isn't being updated anyway while the node is active.

This permits a no-graph lnd mode where no syncing occurs at all.
2021-02-10 09:35:45 -08:00
Olaoluwa Osuntokun
555de44d9f Revert "Merge pull request #4895 from wpaulino/disallow-premature-chan-updates"
This reverts commit 6e6384114c, reversing
changes made to 98ea433271.
2021-02-09 19:55:45 -08:00
Conner Fromknecht
b1fee734ec discovery/sync_manager: remove unneeded markGraphSyncing
AFAICT it's not possible to flip back from bein synced_to_chain, so we
remove the underlying call that could reflect this. The method is moved
into the test file since it's still used to test correctness of other
portions of the flow.
2021-01-29 00:19:48 -08:00
Conner Fromknecht
e42301dee2 lntest: call markGraphSynced from gossipSyncer
Rather than performing this call in the SyncManager, we give each
gossipSyncer the ability to mark the first sync completed. This permits
pinned syncers to contribute towards the rpc-level synced_to_graph
value, allowing the value to be true after the first pinned syncer or
regular syncer complets. Unlinke regular syncers, pinned syncers can
proceed in parallel possibly decreasing the waiting time if consumers
rely on this field before proceeding to load their application.
2021-01-29 00:19:48 -08:00
Conner Fromknecht
fcd5cb625a config: expose gossip.pinned-syncers for conf
The pinned syncer set is exposed as a comma-separated list of pubkeys.
2021-01-29 00:19:47 -08:00
Conner Fromknecht
340414356d discovery: perform initial historical sync for pinned peers 2021-01-29 00:19:47 -08:00
Conner Fromknecht
2f0d56d539 discovery: add support for PinnedSyncers
A pinned syncer is an ActiveSyncer that is configured to always remain
active for the lifetime of the connection. Pinned syncers do not count
towards the total NumActiveSyncer count, which are rotated periodically.

This features allows nodes to more tightly synchronize their routing
tables by ensuring they are always receiving gossip from distinguished
subset of peers.
2021-01-29 00:19:47 -08:00
Conner Fromknecht
9e932f2a64 discovery/sync_manager: Pause/Resume HistoricalSyncTicker
This gives each initial historical syncer an equal amount of time before
being rotated, even if some fail.
2021-01-29 00:19:47 -08:00
Conner Fromknecht
ef0cd82c1f discovery/sync_manager: make setHistoricalSyncer closure 2021-01-29 00:19:46 -08:00
Conner Fromknecht
72fbd1283b discovery/sync_manager: break out IsGraphSynced check 2021-01-29 00:19:46 -08:00
Conner Fromknecht
7c6aa20bd8 discovery: handle err for linter 2021-01-29 00:19:46 -08:00
Wilmer Paulino
7ef1f3f636 discovery: use source of ann upon confirmed channel ann batch
We do this instead of using the source of the AnnounceSignatures
message, as we filter out the source when broadcasting any
announcements, leading to the remote node not receiving our channel
update. Note that this is done more for the sake of correctness and to
address a flake within the integration tests, as channel updates are
sent directly and reliably to channel counterparts.
2021-01-06 13:16:44 -08:00
Wilmer Paulino
00d4e92362 discovery: prevent rebroadcast of premature channel updates
As similarly done with premature channel announcements, we'll no longer
allow premature channel updates to be rebroadcast once mature. This is
no longer necessary as channel announcements that we're not aware of are
usually broadcast to us with their accompanying channel updates.
2021-01-06 12:52:41 -08:00
Wilmer Paulino
871a6f1690 discovery: prevent rebroadcast of previously premature announcements 2020-12-08 15:18:08 -08:00
Wilmer Paulino
a4f33ae63c discovery: adhere to proper channel chunk splitting for ReplyChannelRange 2020-12-08 15:18:07 -08:00
Wilmer Paulino
c5fc7334a4 discovery: limit NumBlocks to best known height for outgoing QueryChannelRange
This is done to ensure we don't receive replies for channels in blocks
not currently known to us, which we wouldn't be able to process.
2020-12-08 15:18:06 -08:00
Olaoluwa Osuntokun
13a2598ded discovery: add new option to toggle gossip rate limiting
In this commit, we add a new option to toggle gossip rate limiting. This
new option can be useful in contexts that require near instant
propagation of gossip messages like integration tests.
2020-11-30 16:38:56 -08:00
Olaoluwa Osuntokun
7e298f1434 Merge pull request #3367 from cfromknecht/batched-graph-updates
Batched graph updates
2020-11-25 18:40:40 -08:00