Add automatic reconnect (with exponential back-off) to the socket backend.
If the connection is lost while pusing a change, try to reconnect.
Handle edge-cases that might come up depending on when the disconnect
happaned.
Currently the parameters are hardcoded: it will retry 5 times,
on the first retry it will wait 5 seconds before reconnecting.
There is an exponential backoff of 1.5, so on the fifth try it
will wait about 25 seconds.
This is particularly useful when conencting over Tor, as transient
interruptions are fairly common there.
Add two optional arguments to `backup-cli server`:
- `--log-mode`: either `plain` (simply print the log message) or `systemd`
(prefix log message with systemd log level, useful for running as a
a service)
- `--log-level`: minimum level for messages to be logged, from `DEBUG`
to `CRITICAL`
Also, move some noisy messages to the debug level.
Support the bracketed `socket:[::]:1234` syntax for IPv6 addresses.
Also, add factor out the socket URI parsing to a function and add
tests for it. As a forward compatibility measure, reject query strings
(`?a=b`) because I intend to use these for parameters such as SOCKS5
proxy (for Tor) in a future patch.
Additional checking was added that a file exists when `create=False`,
this runs into an error with the `test_rewrite()` test. Update the test
to use dummy backend for the test, as it tests a method on the base
class, not any specific backend.
Compaction for the FileBackend involves restoring the DB from the
backup, up to the penultimate state, then taking a snapshot of the
restored DB, and then appending the last change to the backup. The new
backup is then atomically swapped with the old backup.
The special treatment for the last change is necessary as usual due to
the rewind functionality, otherwise we'd be in an un-rewindable state,
which can cause trouble if c-lightning died without committing the
last change.
Caching writes was causing us some issues if the startup of
`lightningd` didn't complete, i.e., the writes would happen on the
live DB, but we'd forget about them. Since we have the backup.lock
file in same directory as the plugin is running in, we no longer have
to defer the writes. This hugely simplifies our logic.
Fixes#155
Changelog-Fixed: backup: The plugin doesn't lose sync anymore if the startup is interrupted
We'll start relying solely on the `backup.lock` file generated by the
`backup-cli`. But we keep the option so we don't automatically break
all configs that simply updated.
Looking up the process parents and searching for `lightnignd` allows us to
kill directly. It also means we no longer have to wait for the RPC to be
enabled to abort.
We were writing logs directly before killing `lightningd` which resulted in
the logs being lost. The subsequent check for the log-lines then would fail
obviously.
Before this change, backup backend was initialized twice - in on_init()
and get_backend() functions. This change leaves only one call in
get_backend() and adds a kwarg determining whether init failure should
be fatal.
Signed-off-by: Michal Rostecki <mrostecki@mailfence.com>
We heavily rely on foreign key constraints to keep the DB consistent and drop
dependent objects such as UTXO set entries when we roll back a block. We also
need to enable the foreign key constraints when restoring otherwise these
dependents are not removed and we run into constraint violations.
The node_factory from pyln-client clears the node directory when provisioning
a new node, which interferes with the backup-cli creation of the initial
snapshot. This just disables the cleaning until we can release the next
pyln-client version
