Commit Graph

455 Commits

Author SHA1 Message Date
Christian Decker
4a5f7a0cce probe: Allow the caller to specify the amount to probe
We used to only probe for a 10sat amount, which allows us to test
basic reachability, but it doesn't say much about whether we can
actually pay for something reasonable, so this fixes that.

This may require the following SQL statement to be applied to the
database if you are upgrading from a prior version:

```sql
ALTER TABLE probes ADD amount INTEGER;
```

This is not required if you start with a new database.
2021-03-25 14:47:17 +01:00
Gálli Zoltán
a7a0007dce rebalance: handle HTLC timeout
In rare cases, HTLCs can be stuck for days. I don't know why it's happening, maybe it would worth investigating the situation.
2021-02-26 11:28:07 +01:00
Wladimir J. van der Laan
d59eb41e3f backup: Improve logging configurability for socket server
Add two optional arguments to `backup-cli server`:

- `--log-mode`: either `plain` (simply print the log message) or `systemd`
  (prefix log message with systemd log level, useful for running as a
  a service)

- `--log-level`: minimum level for messages to be logged, from `DEBUG`
  to `CRITICAL`

Also, move some noisy messages to the debug level.
2021-02-26 10:38:22 +01:00
Wladimir J. van der Laan
f66af8d2cf backup: Add section on backup compaction to README 2021-02-26 10:36:29 +01:00
Wladimir J. van der Laan
eada16c688 backup: Add support for Tor
Add support for connecting to the backup server through Tor, via its
SOCKS5 proxy.

See "Usage with Tor" in remote.md for documentation.
2021-02-26 10:35:48 +01:00
Rusty Russell
c8be8a11d7 currencyrate: fix what happens when we get an error.
root@ubuntu-s-4vcpu-8gb-nyc3-01:~# l2-cli currencyrates USD
{
   "code": -32600,
   "message": "Error while processing currencyrates: argument of type 'int' is not iterable",
   "traceback": "Traceback (most recent call last):\n  File \"/usr/local/lib/python3.8/dist-packages/pyln/client/plugin.py\", line 621, in _dispatch_request\n    result = self._exec_func(method.func, request)\n  File \"/usr/local/lib/python3.8/dist-packages/pyln/client/plugin.py\", line 603, in _exec_func\n    return func(*ba.args, **ba.kwargs)\n  File \"/root/plugins/currencyrate/currencyrate.py\", line 121, in currencyrates\n    return get_rates(plugin, currency.upper())\n  File \"/usr/local/lib/python3.8/dist-packages/cachetools/decorators.py\", line 22, in wrapper\n    v = func(*args, **kwargs)\n  File \"/root/plugins/currencyrate/currencyrate.py\", line 110, in get_rates\n    r = get_currencyrate(plugin, currency, s.urlformat, s.replymembers)\n  File \"/root/plugins/currencyrate/currencyrate.py\", line 71, in get_currencyrate\n    r = requests_retry_session(retries=5, status_forcelist=(404)).get(url, proxies=plugin.proxies)\n  File \"/usr/lib/python3/dist-packages/requests/sessions.py\", line 543, in get\n    return self.request('GET', url, **kwargs)\n  File \"/usr/lib/python3/dist-packages/requests/sessions.py\", line 530, in request\n    resp = self.send(prep, **send_kwargs)\n  File \"/usr/lib/python3/dist-packages/requests/sessions.py\", line 643, in send\n    r = adapter.send(request, **kwargs)\n  File \"/usr/lib/python3/dist-packages/requests/adapters.py\", line 439, in send\n    resp = conn.urlopen(\n  File \"/usr/lib/python3/dist-packages/urllib3/connectionpool.py\", line 803, in urlopen\n    if retries.is_retry(method, response.status, has_retry_after):\n  File \"/usr/lib/python3/dist-packages/urllib3/util/retry.py\", line 339, in is_retry\n    if self.status_forcelist and status_code in self.status_forcelist:\nTypeError: argument of type 'int' is not iterable\n"
}

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-02-17 07:37:20 +01:00
Gálli Zoltán
38b03cdddc rebalance: channel wo scid
A channel can exist without a short_channel_id, in the CHANNELD_AWAITING_LOCKIN state, while the funding transaction is sent but not yet included in a block.

Running `rebalanceall` while opening a channel could throw an exception:
```
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py: Exception in thread Thread-5:
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py: Traceback (most recent call last):
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py:   File \"/usr/lib/python3.6/threading.py\", line 916, in _bootstrap_inner
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py:     self.run()
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py:   File \"/usr/lib/python3.6/threading.py\", line 864, in run
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py:     self._target(*self._args, **self._kwargs)
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py:   File \"/home/user/plugins/rebalance/rebalance.py\", line 475, in rebalanceall_thread
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py:     result = maybe_rebalance_once(plugin, failed_pairs)
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py:   File \"/home/user/plugins/rebalance/rebalance.py\", line 443, in maybe_rebalance_once
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py:     result = maybe_rebalance_pairs(plugin, ch1, ch2, failed_pairs)
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py:   File \"/home/user/plugins/rebalance/rebalance.py\", line 425, in maybe_rebalance_pairs
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py:     wait_for_htlcs(plugin, [scid1, scid2])
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py:   File \"/home/user/plugins/rebalance/rebalance.py\", line 384, in wait_for_htlcs
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py:     if scids is not None and channel['short_channel_id'] not in scids:
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py: KeyError: 'short_channel_id'
```
2021-02-16 10:58:35 +01:00
Antoine Poinsot
04eed3f595 Introduce a Minimum Supported Python Version for Python plugins
Fixes https://github.com/lightningd/plugins/issues/146

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-02-11 13:31:50 +01:00
Antoine Poinsot
3923bc787f README: touch on starting plugins via RPC, and two nits
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-02-11 13:31:50 +01:00
Christian Decker
3be58b2580 paytest: Add description of the paytest protocol 2021-02-11 13:29:37 +01:00
Christian Decker
dc070e8e90 paytest: Create a new plugin to test pay implementations 2021-02-11 13:29:37 +01:00
Aido
defdc28915 summary: Add fees_collected info to output 2021-02-10 15:36:52 +01:00
Christian Decker
139ea7de14 prometheus: Update the pyln-client dependency
Reported-By: <@zefir01>
2021-02-08 18:23:23 +01:00
Gálli Zoltán
d219c0abc1 rebalance: measure and log HTLC settlement time
log messages show how much time need for an HTLC settlement
2021-02-08 14:57:27 +01:00
Wladimir J. van der Laan
cdfcd5a2fe backup: Add support for IPv6 addresses in socket backend
Support the bracketed `socket:[::]:1234` syntax for IPv6 addresses.

Also, add factor out the socket URI parsing to a function and add
tests for it. As a forward compatibility measure, reject query strings
(`?a=b`) because I intend to use these for parameters such as SOCKS5
proxy (for Tor) in a future patch.
2021-02-08 10:31:28 +01:00
Peter Sebastian Nordholt
ca75d7e132 Fix string formating. 2021-02-05 09:05:16 +01:00
Peter Sebastian Nordholt
63232c0fb2 Fix handling of ignored messages in historian.py
Fixes a bug where gossip messages that are ignored by the historian plugin will trigger an exception being printed.
2021-02-05 09:05:16 +01:00
Michael Schmoock
c277f9e10b rebalance: adds a check for no incoming liquidity 2021-02-04 14:54:37 +01:00
Wladimir J. van der Laan
be140a72f7 backup: Use dummy backend in test_rewrite
Additional checking was added that a file exists when `create=False`,
this runs into an error with the `test_rewrite()` test. Update the test
to use dummy backend for the test, as it tests a method on the base
class, not any specific backend.
2021-02-04 13:06:41 +01:00
Wladimir J. van der Laan
7819fef71e backup: Update README for --lightning-dir argument 2021-02-04 13:06:41 +01:00
Wladimir J. van der Laan
4d6d738f80 backup: Update test for optional lightning-dir argument 2021-02-04 13:06:41 +01:00
Wladimir J. van der Laan
804a9bb290 backup: Implement network backup 2021-02-04 13:06:41 +01:00
MrManPew
4f4e30bb49 add path for backyup solution info 2021-02-04 13:06:14 +01:00
Peter Sebastian Nordholt
5de2675beb Avoid out of IndexError from historian-stats
Fixes a bug where `historian-stats` would give IndexError when no announcements or updates where recorded yet.
2021-02-03 18:29:33 +01:00
Peter Sebastian Nordholt
368011cbb4 Fix reference to gossipd in historian.py
`gossipd` was referred to as `parser` resulting in errors.
2021-02-03 18:29:07 +01:00
Michael Schmoock
5555c73208 rebalance: test a full rebalanceall run 2021-02-03 14:17:20 +01:00
Michael Schmoock
5231bfff65 rebalance: speedup by checking for htlcs 2021-02-03 14:17:20 +01:00
Michael Schmoock
cf9b942966 rebalance: fix potential looping mistakes 2021-02-03 14:17:20 +01:00
Michael Schmoock
5e00474390 rebalance: test rebalanceall starts without errors 2021-02-03 14:17:20 +01:00
Michael Schmoock
c6c44aca8e rebalance: test manaual rebalancing 2021-02-03 14:17:20 +01:00
Michael Schmoock
f55df28bcd rebalance: adds a very basic rebalanceall test 2021-02-03 14:17:20 +01:00
Michael Schmoock
239a21ab6b rebalance: autofix code nits 2021-02-03 14:17:20 +01:00
Michael Schmoock
3cbe497976 rebalance: fix division by zero 2021-02-03 14:17:20 +01:00
Peter Sebastian Nordholt
a03557044c Fix missing parser error
Fixes an exception thrown for not having a parser for message type 4103. These messages are now just ignored (maybe should be handled somehow?)
2021-02-03 13:51:12 +01:00
3nprob
c7f0823c14 Add Dockerfile 2021-02-03 13:50:44 +01:00
Michael Schmoock
4a9df896ad backup: fix deadlock by removing the sqlite3 check
I commented out that check and placed a NOTE that future developers
to see upfront putting in RPC stuff in init is potentially bad.
2021-02-01 14:33:46 +01:00
Christian Decker
abbdeabec6 historian: Add a readme 2021-01-22 18:00:50 +01:00
Christian Decker
5236850245 historian: Rename the parser module to gossipd
We were getting a couple of path/import conflicts, renaming just to be
safe.
2021-01-22 18:00:50 +01:00
Christian Decker
a9c14da33b historian: Fix the short channel ID output
It's 2 bytes not 1
2021-01-22 18:00:50 +01:00
Christian Decker
487e927116 historian: Make the GossipStream seekable 2021-01-22 18:00:50 +01:00
Christian Decker
b7f6796315 historian: Make the destination parameter to snapshot load optional
This is a rather long option, and if `lightning-cli` is configured
correctly we should be able to extract this information directly from
it without problems. This adds auto-discovery based on `lightning-cli`
which will to the right thing™
2021-01-22 18:00:50 +01:00
Christian Decker
b5038e526a historian: Ensure parser is called with bytes 2021-01-22 18:00:50 +01:00
Christian Decker
4630eba3f8 historian: Refactor the snapshot message retrieval
We're going to use it in the web interface soon.
2021-01-22 18:00:50 +01:00
Christian Decker
3daa588be0 historian: Remove DATETIME conversion and add timestamp to grouping
Postgres requires the HAVING statement to match a GROUP BY statement and the
DATETIME function doesn't exist on postgres.
2021-01-22 18:00:16 +01:00
Christian Decker
91f7eafabb historian: Use LargeBinary instead of BLOB for portability 2021-01-22 18:00:16 +01:00
Christian Decker
3f5dc35ab1 historian: Add backup command for lnresearch 2021-01-22 18:00:16 +01:00
Christian Decker
474a2011d1 historian: Add assertion on the gossip type when decoding 2021-01-22 18:00:16 +01:00
Christian Decker
b1338ef5d4 historian: Add loading functionality to historian-cli 2021-01-22 18:00:16 +01:00
Christian Decker
2e0963cf40 historian: Switch to nested queries from massive joins
Not only is this more correct, but it's also quicker, since we avoid the huge
joins.
2021-01-22 18:00:16 +01:00
Christian Decker
c22562bd7f historian: Add a command to read a snapshot and hexdump it to stdout 2021-01-22 18:00:16 +01:00