diff --git a/apps/cli/DEPENDENCIES.md b/apps/cli/DEPENDENCIES.md index abc6941..ef2e951 100644 --- a/apps/cli/DEPENDENCIES.md +++ b/apps/cli/DEPENDENCIES.md @@ -1,10 +1,10 @@ # Dependencies -`pisa-cli` has both system-wide and Python dependencies. This document walks you trough how to satisfy them. +`wt_cli` has both system-wide and Python dependencies. This document walks you through how to satisfy them. ## System-wide dependencies -`pisa-cli` has the following system-wide dependencies: +`wt_cli` has the following system-wide dependencies: - `python3` - `pip3` @@ -27,7 +27,7 @@ It is also likely that, if `python3` is installed in our system, the `python` al python3 --version -If `python3` is installed but the `python` alias is not set to it, we should either set it, or use `python3` to run `pisa-cli`. +If `python3` is installed but the `python` alias is not set to it, we should either set it, or use `python3` to run `wt_cli`. Regarding `pip`, we can check what version is installed in our system (if any) by running: @@ -74,7 +74,7 @@ and for `pip3`: ## Python dependencies -`pisa-cli` has the following dependencies (which can be satisfied by using `pip install -r requirements.txt`): +`wt_cli` has the following dependencies (which can be satisfied by using `pip install -r requirements.txt`): - `cryptography` -- `requests` \ No newline at end of file +- `requests` diff --git a/apps/cli/INSTALL.md b/apps/cli/INSTALL.md index 007fcf2..f831f83 100644 --- a/apps/cli/INSTALL.md +++ b/apps/cli/INSTALL.md @@ -1,8 +1,11 @@ # Install -`pisa-cli` has some dependencies that can be satisfied by following [DEPENDENCIES.md](DEPENDENCIES.md). If your system already satisfies the dependencies, you can skip that part. +`wt_cli` has some dependencies that can be satisfied by following [DEPENDENCIES.md](DEPENDENCIES.md). If your system already satisfies the dependencies, you can skip that part. -In order to run `pisa-cli`, you should set your `PYTHONPATH` env variable to include the folder that contains the `apps` folder. You can do so by running: +There are two ways of running `wt_cli`: adding the library to the `PYTHONPATH` env variable, or running it as a module. + +## Modifying `PYTHONPATH` +In order to run `wt_cli`, you should set your `PYTHONPATH` env variable to include the folder that contains the `apps` folder. You can do so by running: export PYTHONPATH=$PYTHONPATH: @@ -10,11 +13,23 @@ For example, for user alice running a UNIX system and having `apps` in her home export PYTHONPATH=$PYTHONPATH:/home/alice/ -You should also include the command in your `.bash_rc` to avoid having to run it every time you open a new terminal. You can do it by running: +You should also include the command in your `.bashrc` to avoid having to run it every time you open a new terminal. You can do it by running: - echo 'export PYTHONPATH=$PYTHONPATH:' >> ~/.bash_rc + echo 'export PYTHONPATH=$PYTHONPATH:' >> ~/.bashrc -Once the `PYTHONPATH` is set, you should be able to run `pisa-cli` straightaway. Try it by running: +Once the `PYTHONPATH` is set, you should be able to run `wt_cli` straightaway. Try it by running: cd /apps/cli - python pisa-cli.py -h \ No newline at end of file + python wt_cli.py -h + +## Running `wt_cli` as a module +Python code can be also run as a module, to do so you need to use `python -m`. From `apps` **parent** directory run: + + python -m apps.cli.wt_cli -h + +Notice that if you run `wt_cli` as a module, you'll need to replace all the calls from `python wt_cli.py ` to `python -m apps.cli.wt_cli ` + +## Modify configuration parameters +If you'd like to modify some of the configuration defaults (such as the user directory, where the logs and appointment receipts will be stored) you can do so in the config file located at: + + /apps/cli/conf.py diff --git a/apps/cli/PISA-API.md b/apps/cli/PISA-API.md index 60038b1..a95697f 100644 --- a/apps/cli/PISA-API.md +++ b/apps/cli/PISA-API.md @@ -2,12 +2,12 @@ ### Disclaimer: Everything in here is experimental and subject to change. -The PISA REST API consists, currently, of two endpoints: `/` and `/check_appointment` +The PISA REST API consists, currently, of two endpoints: `/` and `/get_appointment` `/` is the default endpoint, and is where the appointments should be sent to. `/` accepts `HTTP POST` requests only, with json request body, where data must match the following format: {"locator": l, "start_time": s, "end_time": e, - "dispute_delta": d, "encrypted_blob": eb} + "to_self_delay": d, "encrypted_blob": eb} We'll discuss the parameters one by one in the following: @@ -21,12 +21,19 @@ The to\_self\_delay, `d`, is the time PISA would have to respond with the **pena The encrypted\_blob, `eb`, is a data blob containing the `raw penalty transaction` and it is encrypted using `CHACHA20-POLY1305`. The `encryption key` used by the cipher is the sha256 of the **dispute transaction id**, and the `nonce` is a 12-byte long zero byte array: - sk = sk = sha256(unhexlify(secret)).digest() - nonce = nonce = bytearray(12) # b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' + sk = sha256(unhexlify(secret)).digest() + nonce = bytearray(12) # b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' Finally, the encrypted blob must be hex encoded. `type(eb) = hex encoded str` -The API will return a `application/json` HTTP response code `200/OK` if the appointment is accepted, with the locator encoded in the response text, or a `400/Bad Request` if the appointment is rejected, with the rejection reason encoded in the response text. +The API will return a `application/json` HTTP response code `200/OK` if the appointment is accepted, with the locator encoded in the response text, or a `400/Bad Request` if the appointment is rejected, with the rejection reason encoded in the response text. + +### Alpha release restrictions +The alpha release does not have authentication, payments nor rate limiting, therefore some self imposed restrictions apply: + +- `start_time` should be within the next 6 blocks `[current_time+1, current_time+6]`. +- `end_time` cannot be bigger than (roughtly) a month. That is `4320` blocks on top of `start_time`. +- `encrypted_blob`s are limited to `2 kib`. #### Appointment example @@ -36,21 +43,21 @@ The API will return a `application/json` HTTP response code `200/OK` if the appo "to_self_delay": 20, "encrypted_blob": "6c7687a97e874363e1c2b9a08386125e09ea000a9b4330feb33a5c698265f3565c267554e6fdd7b0544ced026aaab73c255bcc97c18eb9fa704d9cc5f1c83adaf921de7ba62b2b6ddb1bda7775288019ec3708642e738eddc22882abf5b3f4e34ef2d4077ed23e135f7fe22caaec845982918e7df4a3f949cadd2d3e7c541b1dbf77daf64e7ed61531aaa487b468581b5aa7b1da81e2617e351c9d5cf445e3391c3fea4497aaa7ad286552759791b9caa5e4c055d1b38adfceddb1ef2b99e3b467dd0b0b13ce863c1bf6b6f24543c30d"} -# Check appointment +# Get appointment -`/check_appointment` is an endpoint provided to check the status of the appointments sent to PISA. The endpoint is accessible without any type of authentication for now. `/check_appointment` accepts `HTTP GET` requests only, where the data to be provided must be the locator of an appointment. The query must match the following format: +`/get_appointment` is an endpoint provided to check the status of the appointments sent to PISA. The endpoint is accessible without any type of authentication for now. `/get_appointment` accepts `HTTP GET` requests only, where the data to be provided must be the **locator** of an appointment. The query must match the following format: -`http://pisa_server:pisa_port/check_appointment?locator=appointment_locator` +`https://pisa_server:pisa_port/get_appointment?locator=appointment_locator` -### Appointment can be in three states +**Appointment can be in three states**: - `not_found`: meaning the locator is not recognised by the API. This could either mean the locator is wrong, or the appointment has already been fulfilled. -- `being_watched`: the appointment has been accepted by the PISA server and it's being watched at the moment. This stage means that the dispute transaction has not been seen yet, and therefore no justice transaction has been published. -- `dispute_responded`: the dispute was found by the watcher and the corresponding justice transaction has been broadcast by the node. In this stage PISA is actively monitoring until the justice transaction reaches enough confirmations and making sure no fork occurs in the meantime. +- `being_watched`: the appointment has been accepted by the PISA server and it's being watched at the moment. This stage means that the dispute transaction has not been seen yet, and therefore no penalty transaction has been published. +- `dispute_responded`: the dispute was found by the watcher and the corresponding penalty transaction has been broadcast by the node. In this stage PISA is actively monitoring until the penalty transaction reaches enough confirmations and making sure no fork occurs in the meantime. -### Check appointment response formats +### Get appointment response formats -`/check_appointment` will always reply with `json` containing the information about the requested appointment. The structure is as follows: +`/get_appointment` will always reply with `json` containing the information about the requested appointment. The structure is as follows: **not_found** diff --git a/apps/cli/README.md b/apps/cli/README.md index ddea17a..0e5c825 100644 --- a/apps/cli/README.md +++ b/apps/cli/README.md @@ -15,9 +15,8 @@ Refer to [INSTALL.md](INSTALL.md) #### Global options -- `-s, --server`: API server where to send the requests. Defaults to btc.pisa.watch (modifiable in \_\_init\_\_.py) -- `-p, --port` : API port where to send the requests. Defaults to 9814 (modifiable in \_\_init\_\_.py) -- `-d, --debug`: shows debug information and stores it in pisa.log +- `-s, --server`: API server where to send the requests. Defaults to https://teos.pisa.watch (modifiable in conf.py) +- `-p, --port` : API port where to send the requests. Defaults to 443 (modifiable in conf.py) - `-h --help`: shows a list of commands or help for a specific command. #### Commands @@ -36,8 +35,7 @@ This command is used to register appointments to the PISA server. Appointments * "tx_id": tx_id, "start_time": s, "end_time": e, - "dispute_delta": d - } + "to_self_delay": d } `tx` **must** be the raw penalty transaction that will be encrypted before sent to the PISA server. `type(tx) = hex encoded str` @@ -51,6 +49,13 @@ This command is used to register appointments to the PISA server. Appointments * The API will return a `application/json` HTTP response code `200/OK` if the appointment is accepted, with the locator encoded in the response text, or a `400/Bad Request` if the appointment is rejected, with the rejection reason encoded in the response text. +### Alpha release restrictions +The alpha release does not have authentication, payments nor rate limiting, therefore some self imposed restrictions apply: + +- `start_time` should be within the next 6 blocks `[current_time+1, current_time+6]`. +- `end_time` cannot be bigger than (roughtly) a month. That is `4320` blocks on top of `start_time`. +- `encrypted_blob`s are limited to `2 kib`. + #### Usage @@ -65,7 +70,7 @@ if `-f, --file` **is** specified, then the command expects a path to a json file This command is used to get information about an specific appointment from the PISA server. -**Appointment can be in three states** +**Appointment can be in three states:** - `not_found`: meaning the locator is not recognised by the tower. This can either mean the locator is wrong, or the appointment has already been fulfilled (the PISA server does not keep track of completed appointments for now). - `being_watched`: the appointment has been accepted by the PISA server and it's being watched at the moment. This stage means that the dispute transaction has not been seen yet, and therefore no penalty transaction has been broadcast. @@ -118,7 +123,7 @@ or 1. Generate a new dummy appointment. **Note:** this appointment will never be fulfilled (it will eventually expire) since it does not corresopond to a valid transaction. However it can be used to interact with the PISA API. ``` -echo '{"tx": "4615a58815475ab8145b6bb90b1268a0dbb02e344ddd483f45052bec1f15b1951c1ee7f070a0993da395a5ee92ea3a1c184b5ffdb2507164bf1f8c1364155d48bdbc882eee0868ca69864a807f213f538990ad16f56d7dfb28a18e69e3f31ae9adad229e3244073b7d643b4597ec88bf247b9f73f301b0f25ae8207b02b7709c271da98af19f1db276ac48ba64f099644af1ae2c90edb7def5e8589a1bb17cc72ac42ecf07dd29cff91823938fd0d772c2c92b7ab050f8837efd46197c9b2b3f", "tx_id": "0b9510d92a50c1d67c6f7fc5d47908d96b3eccdea093d89bcbaf05bcfebdd951", "start_time": 0, "end_time": 0, "to_self_delay": 20}' > dummy_appointment_data.json + echo '{"tx": "4615a58815475ab8145b6bb90b1268a0dbb02e344ddd483f45052bec1f15b1951c1ee7f070a0993da395a5ee92ea3a1c184b5ffdb2507164bf1f8c1364155d48bdbc882eee0868ca69864a807f213f538990ad16f56d7dfb28a18e69e3f31ae9adad229e3244073b7d643b4597ec88bf247b9f73f301b0f25ae8207b02b7709c271da98af19f1db276ac48ba64f099644af1ae2c90edb7def5e8589a1bb17cc72ac42ecf07dd29cff91823938fd0d772c2c92b7ab050f8837efd46197c9b2b3f", "tx_id": "0b9510d92a50c1d67c6f7fc5d47908d96b3eccdea093d89bcbaf05bcfebdd951", "start_time": 0, "end_time": 0, "to_self_delay": 20}' > dummy_appointment_data.json ``` That will create a json file that follows the appointment data structure filled with dummy data and store it in `dummy_appointment_data.json`. **Note**: You'll need to update the `start_time` and `end_time` to match valid block heights. @@ -139,4 +144,4 @@ echo '{"tx": "4615a58815475ab8145b6bb90b1268a0dbb02e344ddd483f45052bec1f15b1951c ## PISA API -If you wish to read about the underlying API, and how to write your own tool to interact with it, refer to [PISA-API.md](PISA-API.md) +If you wish to read about the underlying API, and how to write your own tool to interact with it, refer to [PISA-API.md](PISA-API.md). diff --git a/apps/cli/help.py b/apps/cli/help.py index 298da7a..030c327 100644 --- a/apps/cli/help.py +++ b/apps/cli/help.py @@ -1,9 +1,9 @@ def help_add_appointment(): return ( "NAME:" - "\tpython pisa-cli add_appointment - Registers a json formatted appointment to the PISA server." + "\tpython wt_cli add_appointment - Registers a json formatted appointment to the PISA server." "\n\nUSAGE:" - "\tpython pisa-cli add_appointment [command options] appointment/path_to_appointment_file" + "\tpython wt_cli add_appointment [command options] appointment/path_to_appointment_file" "\n\nDESCRIPTION:" "\n\n\tRegisters a json formatted appointment to the PISA server." "\n\tif -f, --file *is* specified, then the command expects a path to a json file instead of a json encoded " @@ -17,9 +17,9 @@ def help_add_appointment(): def help_get_appointment(): return ( "NAME:" - "\tpython pisa-cli get_appointment - Gets json formatted data about an appointment from the PISA server." + "\tpython wt_cli get_appointment - Gets json formatted data about an appointment from the PISA server." "\n\nUSAGE:" - "\tpython pisa-cli get_appointment appointment_locator" + "\tpython wt_cli get_appointment appointment_locator" "\n\nDESCRIPTION:" "\n\n\tGets json formatted data about an appointment from the PISA server.\n" ) diff --git a/apps/cli/sample_conf.py b/apps/cli/sample_conf.py index ebe03ed..67396fd 100644 --- a/apps/cli/sample_conf.py +++ b/apps/cli/sample_conf.py @@ -1,9 +1,9 @@ -# PISA-SERVER -DEFAULT_PISA_API_SERVER = "btc.pisa.watch" -DEFAULT_PISA_API_PORT = 9814 +# PISA-WT-SERVER +DEFAULT_PISA_API_SERVER = "https://teos.pisa.watch" +DEFAULT_PISA_API_PORT = 443 -# PISA-CLI -DATA_FOLDER = "~/.pisa_btc/" +# WT-CLI +DATA_FOLDER = "~/.wt_cli/" CLIENT_LOG_FILE = "cli.log" APPOINTMENTS_FOLDER_NAME = "appointment_receipts" diff --git a/apps/cli/wt_cli.py b/apps/cli/wt_cli.py index c9b41b9..0f10733 100644 --- a/apps/cli/wt_cli.py +++ b/apps/cli/wt_cli.py @@ -11,7 +11,7 @@ from uuid import uuid4 from apps.cli import config, LOG_PREFIX from apps.cli.help import help_add_appointment, help_get_appointment -from apps.cli.blob import Blob +from common.blob import Blob import common.cryptographer from common import constants @@ -235,7 +235,7 @@ def post_appointment(data): logger.info("Sending appointment to PISA") try: - add_appointment_endpoint = "http://{}:{}".format(pisa_api_server, pisa_api_port) + add_appointment_endpoint = "{}:{}".format(pisa_api_server, pisa_api_port) return requests.post(url=add_appointment_endpoint, json=json.dumps(data), timeout=5) except ConnectTimeout: @@ -246,6 +246,12 @@ def post_appointment(data): logger.error("Can't connect to PISA API. Server cannot be reached") return None + except requests.exceptions.InvalidSchema: + logger.error("No transport protocol found. Have you missed http(s):// in the server url?") + + except requests.exceptions.Timeout: + logger.error("The request timed out") + def process_post_appointment_response(response): """ @@ -263,7 +269,9 @@ def process_post_appointment_response(response): response_json = response.json() except json.JSONDecodeError: - logger.error("The response was not valid JSON") + logger.error( + "The server returned a non-JSON response", status_code=response.status_code, reason=response.reason + ) return None if response.status_code != constants.HTTP_OK: @@ -337,7 +345,7 @@ def get_appointment(locator): logger.error("The provided locator is not valid", locator=locator) return None - get_appointment_endpoint = "http://{}:{}/get_appointment".format(pisa_api_server, pisa_api_port) + get_appointment_endpoint = "{}:{}/get_appointment".format(pisa_api_server, pisa_api_port) parameters = "?locator={}".format(locator) try: @@ -352,11 +360,17 @@ def get_appointment(locator): logger.error("Can't connect to PISA API. Server cannot be reached") return None + except requests.exceptions.InvalidSchema: + logger.error("No transport protocol found. Have you missed http(s):// in the server url?") + + except requests.exceptions.Timeout: + logger.error("The request timed out") + def show_usage(): return ( "USAGE: " - "\n\tpython pisa-cli.py [global options] command [command options] [arguments]" + "\n\tpython wt_cli.py [global options] command [command options] [arguments]" "\n\nCOMMANDS:" "\n\tadd_appointment \tRegisters a json formatted appointment to the PISA server." "\n\tget_appointment \tGets json formatted data about an appointment from the PISA server." @@ -365,7 +379,7 @@ def show_usage(): "\n\t-s, --server \tAPI server where to send the requests. Defaults to btc.pisa.watch (modifiable in " "__init__.py)" "\n\t-p, --port \tAPI port where to send the requests. Defaults to 9814 (modifiable in __init__.py)" - "\n\t-d, --debug \tshows debug information and stores it in pisa_cli.log" + "\n\t-d, --debug \tshows debug information and stores it in wt_cli.log" "\n\t-h --help \tshows this message." ) diff --git a/common/appointment.py b/common/appointment.py index 7b21471..e376668 100644 --- a/common/appointment.py +++ b/common/appointment.py @@ -2,7 +2,7 @@ import json import struct from binascii import unhexlify -from pisa.encrypted_blob import EncryptedBlob +from common.encrypted_blob import EncryptedBlob class Appointment: @@ -16,7 +16,7 @@ class Appointment: end_time (:mod:`int`): The block height where the tower will stop watching for breaches. to_self_delay (:mod:`int`): The ``to_self_delay`` encoded in the ``csv`` of the ``htlc`` that this appointment is covering. - encrypted_blob (:obj:`EncryptedBlob `): An ``EncryptedBlob`` object + encrypted_blob (:obj:`EncryptedBlob `): An ``EncryptedBlob`` object containing an encrypted penalty transaction. The tower will decrypt it and broadcast the penalty transaction upon seeing a breach on the blockchain. """ diff --git a/apps/cli/blob.py b/common/blob.py similarity index 100% rename from apps/cli/blob.py rename to common/blob.py diff --git a/common/cryptographer.py b/common/cryptographer.py index 6519620..c701f5f 100644 --- a/common/cryptographer.py +++ b/common/cryptographer.py @@ -49,12 +49,12 @@ class Cryptographer: @staticmethod def encrypt(blob, secret, rtype="str"): """ - Encrypts a given :mod:`Blob ` data using ``CHACHA20POLY1305``. + Encrypts a given :mod:`Blob ` data using ``CHACHA20POLY1305``. ``SHA256(secret)`` is used as ``key``, and ``0 (12-byte)`` as ``iv``. Args: - blob (:mod:`Blob `): a ``Blob`` object containing a raw penalty transaction. + blob (:mod:`Blob `): a ``Blob`` object containing a raw penalty transaction. secret (:mod:`str`): a value to used to derive the encryption key. Should be the dispute txid. rtype(:mod:`str`): the return type for the encrypted value. Can be either ``'str'`` or ``'bytes'``. @@ -78,7 +78,7 @@ class Cryptographer: sk = sha256(unhexlify(secret)).digest() nonce = bytearray(12) - logger.info("Encrypting blob", sk=hexlify(sk).decode(), nonce=hexlify(nonce).decode(), blob=blob.data) + logger.debug("Encrypting blob", sk=hexlify(sk).decode(), nonce=hexlify(nonce).decode(), blob=blob.data) # Encrypt the data cipher = ChaCha20Poly1305(sk) @@ -93,12 +93,12 @@ class Cryptographer: # ToDo: #20-test-tx-decrypting-edge-cases def decrypt(encrypted_blob, secret, rtype="str"): """ - Decrypts a given :mod:`EncryptedBlob ` using ``CHACHA20POLY1305``. + Decrypts a given :mod:`EncryptedBlob ` using ``CHACHA20POLY1305``. ``SHA256(secret)`` is used as ``key``, and ``0 (12-byte)`` as ``iv``. Args: - encrypted_blob(:mod:`EncryptedBlob `): an ``EncryptedBlob`` potentially + encrypted_blob(:mod:`EncryptedBlob `): an ``EncryptedBlob`` potentially containing a penalty transaction. secret (:mod:`str`): a value to used to derive the decryption key. Should be the dispute txid. rtype(:mod:`str`): the return type for the decrypted value. Can be either ``'str'`` or ``'bytes'``. diff --git a/pisa/encrypted_blob.py b/common/encrypted_blob.py similarity index 100% rename from pisa/encrypted_blob.py rename to common/encrypted_blob.py diff --git a/common/logger.py b/common/logger.py index b175ebf..136b330 100644 --- a/common/logger.py +++ b/common/logger.py @@ -60,7 +60,7 @@ class Logger: def debug(self, msg, **kwargs): """ - Logs an ``DEBUG`` level message to stdout and file. + Logs a ``DEBUG`` level message to stdout and file. Args: msg (:obj:`str`): the message to be logged. @@ -84,7 +84,7 @@ class Logger: def warning(self, msg, **kwargs): """ - Logs an ``WARNING`` level message to stdout and file. + Logs a ``WARNING`` level message to stdout and file. Args: msg (:obj:`str`): the message to be logged. diff --git a/common/tools.py b/common/tools.py index 0c131da..55dfe09 100644 --- a/common/tools.py +++ b/common/tools.py @@ -129,10 +129,10 @@ def setup_logging(log_file_path, log_name_prefix): # Create the file logger f_logger = logging.getLogger("{}_file_log".format(log_name_prefix)) - f_logger.setLevel(logging.INFO) + f_logger.setLevel(logging.DEBUG) fh = logging.FileHandler(log_file_path) - fh.setLevel(logging.INFO) + fh.setLevel(logging.DEBUG) fh_formatter = logging.Formatter("%(message)s") fh.setFormatter(fh_formatter) f_logger.addHandler(fh) diff --git a/pisa/api.py b/pisa/api.py index e31d0d2..c7b4e20 100644 --- a/pisa/api.py +++ b/pisa/api.py @@ -7,7 +7,6 @@ from pisa import HOST, PORT, LOG_PREFIX from common.logger import Logger from pisa.inspector import Inspector from common.appointment import Appointment -from pisa.block_processor import BlockProcessor from common.constants import HTTP_OK, HTTP_BAD_REQUEST, HTTP_SERVICE_UNAVAILABLE, LOCATOR_LEN_HEX @@ -36,46 +35,54 @@ class API: can be found at :mod:`Errors `. """ - remote_addr = request.environ.get("REMOTE_ADDR") - remote_port = request.environ.get("REMOTE_PORT") + # Getting the real IP if the server is behind a reverse proxy + remote_addr = request.environ.get("HTTP_X_REAL_IP") + if not remote_addr: + remote_addr = request.environ.get("REMOTE_ADDR") - logger.info("Connection accepted", from_addr_port="{}:{}".format(remote_addr, remote_port)) + logger.info("Received add_appointment request", from_addr="{}".format(remote_addr)) - # Check content type once if properly defined - request_data = json.loads(request.get_json()) - inspector = Inspector(self.config) - appointment = inspector.inspect( - request_data.get("appointment"), request_data.get("signature"), request_data.get("public_key") - ) + # FIXME: Logging every request so we can get better understanding of bugs in the alpha + logger.debug("Request details", data="{}".format(request.data)) - error = None - response = None + if request.is_json: + # Check content type once if properly defined + request_data = json.loads(request.get_json()) + inspector = Inspector(self.config) + appointment = inspector.inspect( + request_data.get("appointment"), request_data.get("signature"), request_data.get("public_key") + ) - if type(appointment) == Appointment: - appointment_added, signature = self.watcher.add_appointment(appointment) + error = None + response = None - if appointment_added: - rcode = HTTP_OK - response = {"locator": appointment.locator, "signature": signature} + if type(appointment) == Appointment: + appointment_added, signature = self.watcher.add_appointment(appointment) + + if appointment_added: + rcode = HTTP_OK + response = {"locator": appointment.locator, "signature": signature} + + else: + rcode = HTTP_SERVICE_UNAVAILABLE + error = "appointment rejected" + + elif type(appointment) == tuple: + rcode = HTTP_BAD_REQUEST + error = "appointment rejected. Error {}: {}".format(appointment[0], appointment[1]) else: - rcode = HTTP_SERVICE_UNAVAILABLE - error = "appointment rejected" - - elif type(appointment) == tuple: - rcode = HTTP_BAD_REQUEST - error = "appointment rejected. Error {}: {}".format(appointment[0], appointment[1]) + # We should never end up here, since inspect only returns appointments or tuples. Just in case. + rcode = HTTP_BAD_REQUEST + error = "appointment rejected. Request does not match the standard" else: - # We should never end up here, since inspect only returns appointments or tuples. Just in case. rcode = HTTP_BAD_REQUEST - error = "appointment rejected. Request does not match the standard" + error = "appointment rejected. Request is not json encoded" + response = None logger.info( - "Sending response and disconnecting", - from_addr_port="{}:{}".format(remote_addr, remote_port), - response=response, - error=error, + "Sending response and disconnecting", from_addr="{}".format(remote_addr), response=response, error=error ) if error is None: @@ -83,7 +90,7 @@ class API: else: return jsonify({"error": error}), rcode - # FIXME: THE NEXT THREE API ENDPOINTS ARE FOR TESTING AND SHOULD BE REMOVED / PROPERLY MANAGED BEFORE PRODUCTION! + # FIXME: THE NEXT TWO API ENDPOINTS ARE FOR TESTING AND SHOULD BE REMOVED / PROPERLY MANAGED BEFORE PRODUCTION! # ToDo: #17-add-api-keys def get_appointment(self): """ @@ -102,9 +109,16 @@ class API: - Unknown appointments are flagged as ``not_found``. """ + # Getting the real IP if the server is behind a reverse proxy + remote_addr = request.environ.get("HTTP_X_REAL_IP") + if not remote_addr: + remote_addr = request.environ.get("REMOTE_ADDR") + locator = request.args.get("locator") response = [] + logger.info("Received get_appointment request", from_addr="{}".format(remote_addr), locator=locator) + # ToDo: #15-add-system-monitor if not isinstance(locator, str) or len(locator) != LOCATOR_LEN_HEX: response.append({"locator": locator, "status": "not_found"}) @@ -162,21 +176,6 @@ class API: return response - @staticmethod - def get_block_count(): - """ - Provides the block height of the Watchtower. - - This is a testing endpoint that (most likely) will be removed in production. Its purpose is to give information - to testers about the current block so they can define a dummy appointment without having to run a bitcoin node. - - Returns: - :obj:`dict`: A json encoded dictionary containing the block height. - - """ - - return jsonify({"block_count": BlockProcessor.get_block_count()}) - def start(self): """ This function starts the Flask server used to run the API. Adds all the routes to the functions listed above. @@ -186,7 +185,6 @@ class API: "/": (self.add_appointment, ["POST"]), "/get_appointment": (self.get_appointment, ["GET"]), "/get_all_appointments": (self.get_all_appointments, ["GET"]), - "/get_block_count": (self.get_block_count, ["GET"]), } for url, params in routes.items(): diff --git a/pisa/inspector.py b/pisa/inspector.py index ee5bd10..44ada80 100644 --- a/pisa/inspector.py +++ b/pisa/inspector.py @@ -19,6 +19,10 @@ common.cryptographer.logger = Logger(actor="Cryptographer", log_name_prefix=LOG_ # stored + blacklist if multiple wrong requests are received. +BLOCKS_IN_A_MONTH = 4320 # 4320 = roughly a month in blocks +ENCRYPTED_BLOB_MAX_SIZE_HEX = 2 * 2048 + + class Inspector: """ The :class:`Inspector` class is in charge of verifying that the appointment data provided by the user is correct. @@ -161,7 +165,14 @@ class Inspector: if start_time < block_height: message = "start_time is in the past" else: - message = "start_time is too close to current height" + message = ( + "start_time is too close to current height. " + "Accepted times are: [current_height+1, current_height+6]" + ) + + elif start_time > block_height + 6: + rcode = errors.APPOINTMENT_FIELD_TOO_BIG + message = "start_time is too far in the future. Accepted start times are up to 6 blocks in the future" if message is not None: logger.error(message) @@ -206,6 +217,10 @@ class Inspector: rcode = errors.APPOINTMENT_WRONG_FIELD_TYPE message = "wrong end_time data type ({})".format(t) + elif end_time > block_height + BLOCKS_IN_A_MONTH: # 4320 = roughly a month in blocks + rcode = errors.APPOINTMENT_FIELD_TOO_BIG + message = "end_time should be within the next month (<= current_height + 4320)" + elif start_time >= end_time: rcode = errors.APPOINTMENT_FIELD_TOO_SMALL if start_time > end_time: @@ -258,6 +273,12 @@ class Inspector: rcode = errors.APPOINTMENT_WRONG_FIELD_TYPE message = "wrong to_self_delay data type ({})".format(t) + elif to_self_delay > pow(2, 32): + rcode = errors.APPOINTMENT_FIELD_TOO_BIG + message = "to_self_delay must fit the transaction nLockTime field ({} > {})".format( + to_self_delay, pow(2, 32) + ) + elif to_self_delay < self.config.get("MIN_TO_SELF_DELAY"): rcode = errors.APPOINTMENT_FIELD_TOO_SMALL message = "to_self_delay too small. The to_self_delay should be at least {} (current: {})".format( @@ -301,6 +322,10 @@ class Inspector: rcode = errors.APPOINTMENT_WRONG_FIELD_TYPE message = "wrong encrypted_blob data type ({})".format(t) + elif len(encrypted_blob) > ENCRYPTED_BLOB_MAX_SIZE_HEX: + rcode = errors.APPOINTMENT_FIELD_TOO_BIG + message = "encrypted_blob has to be 2Kib at most (current {})".format(len(encrypted_blob) // 2) + elif re.search(r"^[0-9A-Fa-f]+$", encrypted_blob) is None: rcode = errors.APPOINTMENT_WRONG_FIELD_FORMAT message = "wrong encrypted_blob format ({})".format(encrypted_blob) diff --git a/pisa/responder.py b/pisa/responder.py index 4ae73ab..8be3fd8 100644 --- a/pisa/responder.py +++ b/pisa/responder.py @@ -275,7 +275,6 @@ class Responder: if len(self.trackers) > 0 and block is not None: txids = block.get("tx") - logger.info("List of transactions", txids=txids) if self.last_known_block == block.get("previousblockhash"): self.check_confirmations(txids) diff --git a/pisa/watcher.py b/pisa/watcher.py index 281de92..2efca07 100644 --- a/pisa/watcher.py +++ b/pisa/watcher.py @@ -24,7 +24,7 @@ class Watcher: The :class:`Watcher` keeps track of the accepted appointments in ``appointments`` and, for new received block, checks if any breach has happened by comparing the txids with the appointment locators. If a breach is seen, the - :obj:`EncryptedBlob ` of the corresponding appointment is decrypted and the data + :obj:`EncryptedBlob ` of the corresponding appointment is decrypted and the data is passed to the :obj:`Responder `. If an appointment reaches its end with no breach, the data is simply deleted. @@ -81,7 +81,7 @@ class Watcher: the blockchain (``do_watch``) until ``appointments`` is empty. Once a breach is seen on the blockchain, the :obj:`Watcher` will decrypt the corresponding - :obj:`EncryptedBlob ` and pass the information to the + :obj:`EncryptedBlob ` and pass the information to the :obj:`Responder `. The tower may store multiple appointments with the same ``locator`` to avoid DoS attacks based on data @@ -144,7 +144,6 @@ class Watcher: if len(self.appointments) > 0 and block is not None: txids = block.get("tx") - logger.info("List of transactions", txids=txids) expired_appointments = [ uuid @@ -232,7 +231,7 @@ class Watcher: """ Filters what of the found breaches contain valid transaction data. - The :obj:`Watcher` cannot if a given :obj:`EncryptedBlob ` contains a valid + The :obj:`Watcher` cannot if a given :obj:`EncryptedBlob ` contains a valid transaction until a breach if seen. Blobs that contain arbitrary data are dropped and not sent to the :obj:`Responder `. diff --git a/test/apps/cli/unit/test_wt_cli.py b/test/apps/cli/unit/test_wt_cli.py index b05164e..5fe2699 100644 --- a/test/apps/cli/unit/test_wt_cli.py +++ b/test/apps/cli/unit/test_wt_cli.py @@ -14,7 +14,7 @@ from common.tools import compute_locator from common.appointment import Appointment from common.cryptographer import Cryptographer -from apps.cli.blob import Blob +from common.blob import Blob import apps.cli.wt_cli as wt_cli from test.apps.cli.unit.conftest import get_random_value_hex @@ -38,9 +38,9 @@ dummy_pk_der = dummy_pk.public_bytes( # Replace the key in the module with a key we control for the tests wt_cli.pisa_public_key = dummy_pk # Replace endpoint with dummy one -wt_cli.pisa_api_server = "dummy.com" +wt_cli.pisa_api_server = "https://dummy.com" wt_cli.pisa_api_port = 12345 -pisa_endpoint = "http://{}:{}/".format(wt_cli.pisa_api_server, wt_cli.pisa_api_port) +pisa_endpoint = "{}:{}/".format(wt_cli.pisa_api_server, wt_cli.pisa_api_port) dummy_appointment_request = { "tx": get_random_value_hex(192), diff --git a/test/common/unit/test_appointment.py b/test/common/unit/test_appointment.py index 8087138..f83538f 100644 --- a/test/common/unit/test_appointment.py +++ b/test/common/unit/test_appointment.py @@ -4,7 +4,7 @@ import binascii from pytest import fixture from common.appointment import Appointment -from pisa.encrypted_blob import EncryptedBlob +from common.encrypted_blob import EncryptedBlob from test.pisa.unit.conftest import get_random_value_hex diff --git a/test/pisa/unit/test_blob.py b/test/common/unit/test_blob.py similarity index 92% rename from test/pisa/unit/test_blob.py rename to test/common/unit/test_blob.py index a12de8b..7172330 100644 --- a/test/pisa/unit/test_blob.py +++ b/test/common/unit/test_blob.py @@ -1,6 +1,6 @@ from binascii import unhexlify -from apps.cli.blob import Blob +from common.blob import Blob from test.pisa.unit.conftest import get_random_value_hex diff --git a/test/common/unit/test_cryptographer.py b/test/common/unit/test_cryptographer.py index 728e1fd..02b2727 100644 --- a/test/common/unit/test_cryptographer.py +++ b/test/common/unit/test_cryptographer.py @@ -5,10 +5,10 @@ from cryptography.hazmat.primitives.asymmetric import ec from cryptography.hazmat.primitives import serialization import common.cryptographer -from apps.cli.blob import Blob +from common.blob import Blob from common.logger import Logger from common.cryptographer import Cryptographer -from pisa.encrypted_blob import EncryptedBlob +from common.encrypted_blob import EncryptedBlob from test.common.unit.conftest import get_random_value_hex common.cryptographer.logger = Logger(actor="Cryptographer", log_name_prefix="") diff --git a/test/pisa/unit/test_encrypted_blob.py b/test/common/unit/test_encrypted_blob.py similarity index 89% rename from test/pisa/unit/test_encrypted_blob.py rename to test/common/unit/test_encrypted_blob.py index 64add70..b977dbc 100644 --- a/test/pisa/unit/test_encrypted_blob.py +++ b/test/common/unit/test_encrypted_blob.py @@ -1,4 +1,4 @@ -from pisa.encrypted_blob import EncryptedBlob +from common.encrypted_blob import EncryptedBlob from test.pisa.unit.conftest import get_random_value_hex diff --git a/test/pisa/e2e/test_basic_e2e.py b/test/pisa/e2e/test_basic_e2e.py index 3124e7c..96ceed8 100644 --- a/test/pisa/e2e/test_basic_e2e.py +++ b/test/pisa/e2e/test_basic_e2e.py @@ -1,13 +1,11 @@ import json -import binascii from time import sleep from riemann.tx import Tx from pisa import config from pisa import HOST, PORT from apps.cli import wt_cli -from apps.cli.blob import Blob -from apps.cli import config as cli_conf +from common.blob import Blob import common.cryptographer from common.logger import Logger @@ -27,7 +25,7 @@ common.cryptographer.logger = Logger(actor="Cryptographer", log_name_prefix="") # We'll use wt_cli to add appointments. The expected input format is a list of arguments with a json-encoded # appointment -wt_cli.pisa_api_server = HOST +wt_cli.pisa_api_server = "http://{}".format(HOST) wt_cli.pisa_api_port = PORT # Run pisad diff --git a/test/pisa/unit/conftest.py b/test/pisa/unit/conftest.py index 6766faa..28d282d 100644 --- a/test/pisa/unit/conftest.py +++ b/test/pisa/unit/conftest.py @@ -11,7 +11,7 @@ from cryptography.hazmat.backends import default_backend from cryptography.hazmat.primitives.asymmetric import ec from cryptography.hazmat.primitives import serialization -from apps.cli.blob import Blob +from common.blob import Blob from pisa.responder import TransactionTracker from pisa.tools import bitcoin_cli from pisa.db_manager import DBManager