A channel can exist without a short_channel_id, in the CHANNELD_AWAITING_LOCKIN state, while the funding transaction is sent but not yet included in a block.
Running `rebalanceall` while opening a channel could throw an exception:
```
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py: Exception in thread Thread-5:
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py: Traceback (most recent call last):
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py: File \"/usr/lib/python3.6/threading.py\", line 916, in _bootstrap_inner
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py: self.run()
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py: File \"/usr/lib/python3.6/threading.py\", line 864, in run
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py: self._target(*self._args, **self._kwargs)
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py: File \"/home/user/plugins/rebalance/rebalance.py\", line 475, in rebalanceall_thread
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py: result = maybe_rebalance_once(plugin, failed_pairs)
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py: File \"/home/user/plugins/rebalance/rebalance.py\", line 443, in maybe_rebalance_once
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py: result = maybe_rebalance_pairs(plugin, ch1, ch2, failed_pairs)
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py: File \"/home/user/plugins/rebalance/rebalance.py\", line 425, in maybe_rebalance_pairs
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py: wait_for_htlcs(plugin, [scid1, scid2])
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py: File \"/home/user/plugins/rebalance/rebalance.py\", line 384, in wait_for_htlcs
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py: if scids is not None and channel['short_channel_id'] not in scids:
2021-02-14T09:31:39 UNUSUAL plugin-rebalance.py: KeyError: 'short_channel_id'
```
- rebalances liquidity between channels through circular payments
- automatically selects appropriate channels and executes rebalance
- uses economic fee by default: the rebalance transaction fees are less than the node's default fee, so it is cheaper to fix the liquidity than it can be ruined by transaction forwards
- runs async in the background, sometimes for a very long time (maybe hours)
- tries to rebalance small channels to a 50/50 liquidity ratio and stores remaining liquidity in big channels
- does nothing if there is no cheap enough circular route for rebalance
BOLT#7 requires that nodes "SHOULD accept HTLCs that pay a fee equal to or greater than:
fee_base_msat + ( amount_to_forward * fee_proportional_millionths / 1000000 )".
The code in rebalance.py:setup_routing_fees was not strictly meeting the above requirement, as it was using truncating integer division to calculate the proportional part of the fee, which typically results in a fee that is strictly less than the theoretical, infinite-precision, required fee. Some node implementations may allow rounding down here, but some do not, resulting in sporadic WIRE_FEE_INSUFFICIENT errors.
This commit changes the fee calculation in setup_routing_fees so that it always rounds up to the next whole millisatoshi, thereby eliminating many WIRE_FEE_INSUFFICIENT errors and improving the success rate of the rebalance command.
This will set the correct `direction` on manual route hops (in/out).
Before this was not set, resulting in WIRE_FEE_INSUFFICIENT errors,
if fee was calcualted inconsistently with `direction`.
* doc: cleanup and key-value for optional parameters
* rebalance: cleanup and refinements
* sendinvoiceless: cleanup and refinements
* summary: reflects upcoming changes of pylightning to_approx_str
* rebalance: check peer connection on local channels
The channel selection query didnt filter for channels not
having a short_channel_id yet (unconfirmed ones).
This lead to a KeyError whenever an unconfirmed channel existed.
