Merge pull request #96 from sr-gi/fixes-before-alpha

Fixes before alpha

apps/cli/DEPENDENCIES.md

@@ -1,10 +1,10 @@
 # Dependencies
 
-`pisa-cli` has both system-wide and Python dependencies. This document walks you trough how to satisfy them.
+`wt_cli` has both system-wide and Python dependencies. This document walks you through how to satisfy them.
 
 ## System-wide dependencies
 
-`pisa-cli` has the following system-wide dependencies:
+`wt_cli` has the following system-wide dependencies:
 
 - `python3`
 - `pip3`
@@ -27,7 +27,7 @@ It is also likely that, if `python3` is installed in our system, the `python` al
 
 	python3 --version
 
-If `python3` is installed but the `python` alias is not set to it, we should either set it, or use `python3` to run `pisa-cli`.
+If `python3` is installed but the `python` alias is not set to it, we should either set it, or use `python3` to run `wt_cli`.
 
 Regarding `pip`, we can check what version is installed in our system (if any) by running:
 
@@ -74,7 +74,7 @@ and for `pip3`:
 
 ## Python dependencies
 
-`pisa-cli` has the following dependencies (which can be satisfied by using `pip install -r requirements.txt`):
+`wt_cli` has the following dependencies (which can be satisfied by using `pip install -r requirements.txt`):
 
 - `cryptography`
 - `requests`

apps/cli/INSTALL.md

@@ -1,8 +1,11 @@
 # Install 
 
-`pisa-cli` has some dependencies that can be satisfied by following [DEPENDENCIES.md](DEPENDENCIES.md). If your system already satisfies the dependencies, you can skip that part.
+`wt_cli` has some dependencies that can be satisfied by following [DEPENDENCIES.md](DEPENDENCIES.md). If your system already satisfies the dependencies, you can skip that part.
 
-In order to run `pisa-cli`, you should set your `PYTHONPATH` env variable to include the folder that contains the `apps` folder. You can do so by running:
+There are two ways of running `wt_cli`: adding the library to the `PYTHONPATH` env variable, or running it as a module.
+
+## Modifying `PYTHONPATH`
+In order to run `wt_cli`, you should set your `PYTHONPATH` env variable to include the folder that contains the `apps` folder. You can do so by running:
 
 	export PYTHONPATH=$PYTHONPATH:<absolute_path_to_apps>
 	
@@ -10,11 +13,23 @@ For example, for user alice running a UNIX system and having `apps` in her home
 	
 	export PYTHONPATH=$PYTHONPATH:/home/alice/
 	
-You should also include the command in your `.bash_rc` to avoid having to run it every time you open a new terminal. You can do it by running:
+You should also include the command in your `.bashrc` to avoid having to run it every time you open a new terminal. You can do it by running:
 
-	echo 'export PYTHONPATH=$PYTHONPATH:<absolute_path_to_apps>' >> ~/.bash_rc
+	echo 'export PYTHONPATH=$PYTHONPATH:<absolute_path_to_apps>' >> ~/.bashrc
 	
-Once the `PYTHONPATH` is set, you should be able to run `pisa-cli` straightaway. Try it by running:
+Once the `PYTHONPATH` is set, you should be able to run `wt_cli` straightaway. Try it by running:
 
 	cd <absolute_path_to_apps>/apps/cli
-	python pisa-cli.py -h
+	python wt_cli.py -h
+	
+## Running `wt_cli` as a module
+Python code can be also run as a module, to do so you need to use `python -m`. From `apps` **parent** directory run:
+
+    python -m apps.cli.wt_cli -h
+    
+Notice that if you run `wt_cli` as a module, you'll need to replace all the calls from `python wt_cli.py <argument>` to `python -m apps.cli.wt_cli <argument>` 
+
+## Modify configuration parameters
+If you'd like to modify some of the configuration defaults (such as the user directory, where the logs and appointment receipts will be stored) you can do so in the config file located at:
+
+	 <absolute_path_to_apps>/apps/cli/conf.py

apps/cli/PISA-API.md

@@ -2,12 +2,12 @@
 
 ### Disclaimer: Everything in here is experimental and subject to change.
 
-The PISA REST API consists, currently, of two endpoints: `/` and `/check_appointment`
+The PISA REST API consists, currently, of two endpoints: `/` and `/get_appointment`
 
 `/` is the default endpoint, and is where the appointments should be sent to. `/` accepts `HTTP POST` requests only, with json request body, where data must match the following format:
 
 	{"locator": l, "start_time": s, "end_time": e, 
-	"dispute_delta": d, "encrypted_blob": eb}
+	"to_self_delay": d, "encrypted_blob": eb}
 	
 We'll discuss the parameters one by one in the following: 
 	
@@ -21,13 +21,20 @@ The to\_self\_delay, `d`, is the time PISA would have to respond with the **pena
 
 The encrypted\_blob, `eb`, is a data blob containing the `raw penalty transaction` and it is encrypted using `CHACHA20-POLY1305`. The `encryption key` used by the cipher is the sha256 of the **dispute transaction id**, and the `nonce` is a 12-byte long zero byte array:
 
-	sk = sk = sha256(unhexlify(secret)).digest()
+	sk = sha256(unhexlify(secret)).digest()
-	nonce = nonce = bytearray(12) # b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
+	nonce = bytearray(12) # b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
 	
 Finally, the encrypted blob must be hex encoded. `type(eb) = hex encoded str`
 
 The API will return a `application/json` HTTP response code `200/OK` if the appointment is accepted, with the locator encoded in the response text, or a `400/Bad Request` if the appointment is rejected, with the rejection reason encoded in the response text.
 
+### Alpha release restrictions
+The alpha release does not have authentication, payments nor rate limiting, therefore some self imposed restrictions apply:
+
+- `start_time` should be within the next 6 blocks `[current_time+1, current_time+6]`.
+- `end_time` cannot be bigger than (roughtly) a month. That is `4320` blocks on top of `start_time`.
+- `encrypted_blob`s are limited to `2 kib`. 
+
 #### Appointment example
 
 	{"locator": "3c3375883f01027e5ca14f9760a8b853824ca4ebc0258c00e7fae4bae2571a80", 
@@ -36,21 +43,21 @@ The API will return a `application/json` HTTP response code `200/OK` if the appo
 	"to_self_delay": 20, 
 	"encrypted_blob": "6c7687a97e874363e1c2b9a08386125e09ea000a9b4330feb33a5c698265f3565c267554e6fdd7b0544ced026aaab73c255bcc97c18eb9fa704d9cc5f1c83adaf921de7ba62b2b6ddb1bda7775288019ec3708642e738eddc22882abf5b3f4e34ef2d4077ed23e135f7fe22caaec845982918e7df4a3f949cadd2d3e7c541b1dbf77daf64e7ed61531aaa487b468581b5aa7b1da81e2617e351c9d5cf445e3391c3fea4497aaa7ad286552759791b9caa5e4c055d1b38adfceddb1ef2b99e3b467dd0b0b13ce863c1bf6b6f24543c30d"}
 	
-# Check appointment
+# Get appointment
 	
-`/check_appointment` is an endpoint provided to check the status of the appointments sent to PISA. The endpoint is accessible without any type of authentication for now. `/check_appointment` accepts `HTTP GET` requests only, where the data to be provided must be the locator of an appointment. The query must match the following format:
+`/get_appointment` is an endpoint provided to check the status of the appointments sent to PISA. The endpoint is accessible without any type of authentication for now. `/get_appointment` accepts `HTTP GET` requests only, where the data to be provided must be the **locator** of an appointment. The query must match the following format:
 
-`http://pisa_server:pisa_port/check_appointment?locator=appointment_locator`
+`https://pisa_server:pisa_port/get_appointment?locator=appointment_locator`
 
-### Appointment can be in three states
+**Appointment can be in three states**:
 
 - `not_found`: meaning the locator is not recognised by the API. This could either mean the locator is wrong, or the appointment has already been fulfilled.
-- `being_watched`: the appointment has been accepted by the PISA server and it's being watched at the moment. This stage means that the dispute transaction has not been seen yet, and therefore no justice transaction has been published.
+- `being_watched`: the appointment has been accepted by the PISA server and it's being watched at the moment. This stage means that the dispute transaction has not been seen yet, and therefore no penalty transaction has been published.
-- `dispute_responded`: the dispute was found by the watcher and the corresponding justice transaction has been broadcast by the node. In this stage PISA is actively monitoring until the justice transaction reaches enough confirmations and making sure no fork occurs in the meantime.
+- `dispute_responded`: the dispute was found by the watcher and the corresponding penalty transaction has been broadcast by the node. In this stage PISA is actively monitoring until the penalty transaction reaches enough confirmations and making sure no fork occurs in the meantime.
 
-### Check appointment response formats
+### Get appointment response formats
 
-`/check_appointment` will always reply with `json` containing the information about the requested appointment. The structure is as follows:
+`/get_appointment` will always reply with `json` containing the information about the requested appointment. The structure is as follows:
 
 **not_found**
 

apps/cli/README.md

@@ -15,9 +15,8 @@ Refer to [INSTALL.md](INSTALL.md)
 	
 #### Global options
 
-- `-s, --server`:	API server where to send the requests. Defaults to btc.pisa.watch (modifiable in \_\_init\_\_.py)
+- `-s, --server`:	API server where to send the requests. Defaults to https://teos.pisa.watch (modifiable in conf.py)
-- `-p, --port` :	API port where to send the requests. Defaults to 9814 (modifiable in \_\_init\_\_.py)
+- `-p, --port` :	API port where to send the requests. Defaults to 443 (modifiable in conf.py)
-- `-d, --debug`: 	shows debug information and stores it in pisa.log
 - `-h --help`: 	shows a list of commands or help for a specific command.
 
 #### Commands
@@ -36,8 +35,7 @@ This command is used to register appointments to the PISA server. Appointments *
 	  "tx_id": tx_id,
 	  "start_time": s,
 	  "end_time": e,
-	  "dispute_delta": d
+	  "to_self_delay": d }
-	}
 	
 `tx` **must** be the raw penalty transaction that will be encrypted before sent to the PISA server. `type(tx) = hex encoded str`
 
@@ -51,6 +49,13 @@ This command is used to register appointments to the PISA server. Appointments *
 
 The API will return a `application/json` HTTP response code `200/OK` if the appointment is accepted, with the locator encoded in the response text, or a `400/Bad Request` if the appointment is rejected, with the rejection reason encoded in the response text. 
 
+### Alpha release restrictions
+The alpha release does not have authentication, payments nor rate limiting, therefore some self imposed restrictions apply:
+
+- `start_time` should be within the next 6 blocks `[current_time+1, current_time+6]`.
+- `end_time` cannot be bigger than (roughtly) a month. That is `4320` blocks on top of `start_time`.
+- `encrypted_blob`s are limited to `2 kib`.
+
 
 #### Usage
 
@@ -65,7 +70,7 @@ if `-f, --file` **is** specified, then the command expects a path to a json file
 
  This command is used to get information about an specific appointment from the PISA server.	
 
-**Appointment can be in three states**
+**Appointment can be in three states:**
 
 - `not_found`: meaning the locator is not recognised by the tower. This can either mean the locator is wrong, or the appointment has already been fulfilled (the PISA server does not keep track of completed appointments for now).
 - `being_watched`: the appointment has been accepted by the PISA server and it's being watched at the moment. This stage means that the dispute transaction has not been seen yet, and therefore no penalty transaction has been broadcast.
@@ -118,7 +123,7 @@ or
 1. Generate a new dummy appointment. **Note:** this appointment will never be fulfilled (it will eventually expire) since it does not corresopond to a valid transaction. However it can be used to interact with the PISA API.
 
     ```
-echo '{"tx": "4615a58815475ab8145b6bb90b1268a0dbb02e344ddd483f45052bec1f15b1951c1ee7f070a0993da395a5ee92ea3a1c184b5ffdb2507164bf1f8c1364155d48bdbc882eee0868ca69864a807f213f538990ad16f56d7dfb28a18e69e3f31ae9adad229e3244073b7d643b4597ec88bf247b9f73f301b0f25ae8207b02b7709c271da98af19f1db276ac48ba64f099644af1ae2c90edb7def5e8589a1bb17cc72ac42ecf07dd29cff91823938fd0d772c2c92b7ab050f8837efd46197c9b2b3f", "tx_id": "0b9510d92a50c1d67c6f7fc5d47908d96b3eccdea093d89bcbaf05bcfebdd951", "start_time": 0, "end_time": 0, "to_self_delay": 20}' > dummy_appointment_data.json
+	echo '{"tx": "4615a58815475ab8145b6bb90b1268a0dbb02e344ddd483f45052bec1f15b1951c1ee7f070a0993da395a5ee92ea3a1c184b5ffdb2507164bf1f8c1364155d48bdbc882eee0868ca69864a807f213f538990ad16f56d7dfb28a18e69e3f31ae9adad229e3244073b7d643b4597ec88bf247b9f73f301b0f25ae8207b02b7709c271da98af19f1db276ac48ba64f099644af1ae2c90edb7def5e8589a1bb17cc72ac42ecf07dd29cff91823938fd0d772c2c92b7ab050f8837efd46197c9b2b3f", "tx_id": "0b9510d92a50c1d67c6f7fc5d47908d96b3eccdea093d89bcbaf05bcfebdd951", "start_time": 0, "end_time": 0, "to_self_delay": 20}' > dummy_appointment_data.json
     ```
 
     That will create a json file that follows the appointment data structure filled with dummy data and store it in `dummy_appointment_data.json`. **Note**: You'll need to update the `start_time` and `end_time` to match valid block heights.
@@ -139,4 +144,4 @@ echo '{"tx": "4615a58815475ab8145b6bb90b1268a0dbb02e344ddd483f45052bec1f15b1951c
 
 ## PISA API	
 
-If you wish to read about the underlying API, and how to write your own tool to interact with it, refer to [PISA-API.md](PISA-API.md)
+If you wish to read about the underlying API, and how to write your own tool to interact with it, refer to [PISA-API.md](PISA-API.md).

apps/cli/help.py

@@ -1,9 +1,9 @@
 def help_add_appointment():
     return (
         "NAME:"
-        "\tpython pisa-cli add_appointment - Registers a json formatted appointment to the PISA server."
+        "\tpython wt_cli add_appointment - Registers a json formatted appointment to the PISA server."
         "\n\nUSAGE:"
-        "\tpython pisa-cli add_appointment [command options] appointment/path_to_appointment_file"
+        "\tpython wt_cli add_appointment [command options] appointment/path_to_appointment_file"
         "\n\nDESCRIPTION:"
         "\n\n\tRegisters a json formatted appointment to the PISA server."
         "\n\tif -f, --file *is* specified, then the command expects a path to a json file instead of a json encoded "
@@ -17,9 +17,9 @@ def help_add_appointment():
 def help_get_appointment():
     return (
         "NAME:"
-        "\tpython pisa-cli get_appointment - Gets json formatted data about an appointment from the PISA server."
+        "\tpython wt_cli get_appointment - Gets json formatted data about an appointment from the PISA server."
         "\n\nUSAGE:"
-        "\tpython pisa-cli get_appointment appointment_locator"
+        "\tpython wt_cli get_appointment appointment_locator"
         "\n\nDESCRIPTION:"
         "\n\n\tGets json formatted data about an appointment from the PISA server.\n"
     )

apps/cli/sample_conf.py

@@ -1,9 +1,9 @@
-# PISA-SERVER
+# PISA-WT-SERVER
-DEFAULT_PISA_API_SERVER = "btc.pisa.watch"
+DEFAULT_PISA_API_SERVER = "https://teos.pisa.watch"
-DEFAULT_PISA_API_PORT = 9814
+DEFAULT_PISA_API_PORT = 443
 
-# PISA-CLI
+# WT-CLI
-DATA_FOLDER = "~/.pisa_btc/"
+DATA_FOLDER = "~/.wt_cli/"
 
 CLIENT_LOG_FILE = "cli.log"
 APPOINTMENTS_FOLDER_NAME = "appointment_receipts"

apps/cli/wt_cli.py

@@ -11,7 +11,7 @@ from uuid import uuid4
 
 from apps.cli import config, LOG_PREFIX
 from apps.cli.help import help_add_appointment, help_get_appointment
-from apps.cli.blob import Blob
+from common.blob import Blob
 
 import common.cryptographer
 from common import constants
@@ -235,7 +235,7 @@ def post_appointment(data):
     logger.info("Sending appointment to PISA")
 
     try:
-        add_appointment_endpoint = "http://{}:{}".format(pisa_api_server, pisa_api_port)
+        add_appointment_endpoint = "{}:{}".format(pisa_api_server, pisa_api_port)
         return requests.post(url=add_appointment_endpoint, json=json.dumps(data), timeout=5)
 
     except ConnectTimeout:
@@ -246,6 +246,12 @@ def post_appointment(data):
         logger.error("Can't connect to PISA API. Server cannot be reached")
         return None
 
+    except requests.exceptions.InvalidSchema:
+        logger.error("No transport protocol found. Have you missed http(s):// in the server url?")
+
+    except requests.exceptions.Timeout:
+        logger.error("The request timed out")
+
 
 def process_post_appointment_response(response):
     """
@@ -263,7 +269,9 @@ def process_post_appointment_response(response):
         response_json = response.json()
 
     except json.JSONDecodeError:
-        logger.error("The response was not valid JSON")
+        logger.error(
+            "The server returned a non-JSON response", status_code=response.status_code, reason=response.reason
+        )
         return None
 
     if response.status_code != constants.HTTP_OK:
@@ -337,7 +345,7 @@ def get_appointment(locator):
         logger.error("The provided locator is not valid", locator=locator)
         return None
 
-    get_appointment_endpoint = "http://{}:{}/get_appointment".format(pisa_api_server, pisa_api_port)
+    get_appointment_endpoint = "{}:{}/get_appointment".format(pisa_api_server, pisa_api_port)
     parameters = "?locator={}".format(locator)
 
     try:
@@ -352,11 +360,17 @@ def get_appointment(locator):
         logger.error("Can't connect to PISA API. Server cannot be reached")
         return None
 
+    except requests.exceptions.InvalidSchema:
+        logger.error("No transport protocol found. Have you missed http(s):// in the server url?")
+
+    except requests.exceptions.Timeout:
+        logger.error("The request timed out")
+
 
 def show_usage():
     return (
         "USAGE: "
-        "\n\tpython pisa-cli.py [global options] command [command options] [arguments]"
+        "\n\tpython wt_cli.py [global options] command [command options] [arguments]"
         "\n\nCOMMANDS:"
         "\n\tadd_appointment \tRegisters a json formatted appointment to the PISA server."
         "\n\tget_appointment \tGets json formatted data about an appointment from the PISA server."
@@ -365,7 +379,7 @@ def show_usage():
         "\n\t-s, --server \tAPI server where to send the requests. Defaults to btc.pisa.watch (modifiable in "
         "__init__.py)"
         "\n\t-p, --port \tAPI port where to send the requests. Defaults to 9814 (modifiable in __init__.py)"
-        "\n\t-d, --debug \tshows debug information and stores it in pisa_cli.log"
+        "\n\t-d, --debug \tshows debug information and stores it in wt_cli.log"
         "\n\t-h --help \tshows this message."
     )
 

common/appointment.py

@@ -2,7 +2,7 @@ import json
 import struct
 from binascii import unhexlify
 
-from pisa.encrypted_blob import EncryptedBlob
+from common.encrypted_blob import EncryptedBlob
 
 
 class Appointment:
@@ -16,7 +16,7 @@ class Appointment:
         end_time (:mod:`int`): The block height where the tower will stop watching for breaches.
         to_self_delay (:mod:`int`): The ``to_self_delay`` encoded in the ``csv`` of the ``htlc`` that this appointment is
             covering.
-        encrypted_blob (:obj:`EncryptedBlob <pisa.encrypted_blob.EncryptedBlob>`): An ``EncryptedBlob`` object
+        encrypted_blob (:obj:`EncryptedBlob <common.encrypted_blob.EncryptedBlob>`): An ``EncryptedBlob`` object
             containing an encrypted penalty transaction. The tower will decrypt it and broadcast the penalty transaction
             upon seeing a breach on the blockchain.
     """

common/cryptographer.py

@@ -49,12 +49,12 @@ class Cryptographer:
     @staticmethod
     def encrypt(blob, secret, rtype="str"):
         """
-        Encrypts a given :mod:`Blob <apps.cli.blob.Blob>` data using ``CHACHA20POLY1305``.
+        Encrypts a given :mod:`Blob <common.cli.blob.Blob>` data using ``CHACHA20POLY1305``.
 
         ``SHA256(secret)`` is used as ``key``, and ``0 (12-byte)`` as ``iv``.
 
         Args:
-              blob (:mod:`Blob <apps.cli.blob.Blob>`): a ``Blob`` object containing a raw penalty transaction.
+              blob (:mod:`Blob <common.cli.blob.Blob>`): a ``Blob`` object containing a raw penalty transaction.
               secret (:mod:`str`): a value to used to derive the encryption key. Should be the dispute txid.
               rtype(:mod:`str`): the return type for the encrypted value. Can be either ``'str'`` or ``'bytes'``.
 
@@ -78,7 +78,7 @@ class Cryptographer:
         sk = sha256(unhexlify(secret)).digest()
         nonce = bytearray(12)
 
-        logger.info("Encrypting blob", sk=hexlify(sk).decode(), nonce=hexlify(nonce).decode(), blob=blob.data)
+        logger.debug("Encrypting blob", sk=hexlify(sk).decode(), nonce=hexlify(nonce).decode(), blob=blob.data)
 
         # Encrypt the data
         cipher = ChaCha20Poly1305(sk)
@@ -93,12 +93,12 @@ class Cryptographer:
     # ToDo: #20-test-tx-decrypting-edge-cases
     def decrypt(encrypted_blob, secret, rtype="str"):
         """
-        Decrypts a given :mod:`EncryptedBlob <pisa.encrypted_blob.EncryptedBlob>` using ``CHACHA20POLY1305``.
+        Decrypts a given :mod:`EncryptedBlob <common.encrypted_blob.EncryptedBlob>` using ``CHACHA20POLY1305``.
 
         ``SHA256(secret)`` is used as ``key``, and ``0 (12-byte)`` as ``iv``.
 
         Args:
-              encrypted_blob(:mod:`EncryptedBlob <pisa.encrypted_blob.EncryptedBlob>`): an ``EncryptedBlob`` potentially
+              encrypted_blob(:mod:`EncryptedBlob <comnmon.encrypted_blob.EncryptedBlob>`): an ``EncryptedBlob`` potentially
                 containing a penalty transaction.
               secret (:mod:`str`): a value to used to derive the decryption key. Should be the dispute txid.
               rtype(:mod:`str`): the return type for the decrypted value. Can be either ``'str'`` or ``'bytes'``.

common/logger.py

@@ -60,7 +60,7 @@ class Logger:
 
     def debug(self, msg, **kwargs):
         """
-        Logs an ``DEBUG`` level message to stdout and file.
+        Logs a ``DEBUG`` level message to stdout and file.
 
         Args:
              msg (:obj:`str`): the message to be logged.
@@ -84,7 +84,7 @@ class Logger:
 
     def warning(self, msg, **kwargs):
         """
-        Logs an ``WARNING`` level message to stdout and file.
+        Logs a ``WARNING`` level message to stdout and file.
 
         Args:
              msg (:obj:`str`): the message to be logged.

common/tools.py

@@ -129,10 +129,10 @@ def setup_logging(log_file_path, log_name_prefix):
 
     # Create the file logger
     f_logger = logging.getLogger("{}_file_log".format(log_name_prefix))
-    f_logger.setLevel(logging.INFO)
+    f_logger.setLevel(logging.DEBUG)
 
     fh = logging.FileHandler(log_file_path)
-    fh.setLevel(logging.INFO)
+    fh.setLevel(logging.DEBUG)
     fh_formatter = logging.Formatter("%(message)s")
     fh.setFormatter(fh_formatter)
     f_logger.addHandler(fh)

pisa/api.py

@@ -7,7 +7,6 @@ from pisa import HOST, PORT, LOG_PREFIX
 from common.logger import Logger
 from pisa.inspector import Inspector
 from common.appointment import Appointment
-from pisa.block_processor import BlockProcessor
 
 from common.constants import HTTP_OK, HTTP_BAD_REQUEST, HTTP_SERVICE_UNAVAILABLE, LOCATOR_LEN_HEX
 
@@ -36,46 +35,54 @@ class API:
             can be found at :mod:`Errors <pisa.errors>`.
         """
 
-        remote_addr = request.environ.get("REMOTE_ADDR")
+        # Getting the real IP if the server is behind a reverse proxy
-        remote_port = request.environ.get("REMOTE_PORT")
+        remote_addr = request.environ.get("HTTP_X_REAL_IP")
+        if not remote_addr:
+            remote_addr = request.environ.get("REMOTE_ADDR")
 
-        logger.info("Connection accepted", from_addr_port="{}:{}".format(remote_addr, remote_port))
+        logger.info("Received add_appointment request", from_addr="{}".format(remote_addr))
 
-        # Check content type once if properly defined
+        # FIXME: Logging every request so we can get better understanding of bugs in the alpha
-        request_data = json.loads(request.get_json())
+        logger.debug("Request details", data="{}".format(request.data))
-        inspector = Inspector(self.config)
-        appointment = inspector.inspect(
-            request_data.get("appointment"), request_data.get("signature"), request_data.get("public_key")
-        )
 
-        error = None
+        if request.is_json:
-        response = None
+            # Check content type once if properly defined
+            request_data = json.loads(request.get_json())
+            inspector = Inspector(self.config)
+            appointment = inspector.inspect(
+                request_data.get("appointment"), request_data.get("signature"), request_data.get("public_key")
+            )
 
-        if type(appointment) == Appointment:
+            error = None
-            appointment_added, signature = self.watcher.add_appointment(appointment)
+            response = None
 
-            if appointment_added:
+            if type(appointment) == Appointment:
-                rcode = HTTP_OK
+                appointment_added, signature = self.watcher.add_appointment(appointment)
-                response = {"locator": appointment.locator, "signature": signature}
+
+                if appointment_added:
+                    rcode = HTTP_OK
+                    response = {"locator": appointment.locator, "signature": signature}
+
+                else:
+                    rcode = HTTP_SERVICE_UNAVAILABLE
+                    error = "appointment rejected"
+
+            elif type(appointment) == tuple:
+                rcode = HTTP_BAD_REQUEST
+                error = "appointment rejected. Error {}: {}".format(appointment[0], appointment[1])
 
             else:
-                rcode = HTTP_SERVICE_UNAVAILABLE
+                # We  should never end up here, since inspect only returns appointments or tuples. Just in case.
-                error = "appointment rejected"
+                rcode = HTTP_BAD_REQUEST
-
+                error = "appointment rejected. Request does not match the standard"
-        elif type(appointment) == tuple:
-            rcode = HTTP_BAD_REQUEST
-            error = "appointment rejected. Error {}: {}".format(appointment[0], appointment[1])
 
         else:
-            # We  should never end up here, since inspect only returns appointments or tuples. Just in case.
             rcode = HTTP_BAD_REQUEST
-            error = "appointment rejected. Request does not match the standard"
+            error = "appointment rejected. Request is not json encoded"
+            response = None
 
         logger.info(
-            "Sending response and disconnecting",
+            "Sending response and disconnecting", from_addr="{}".format(remote_addr), response=response, error=error
-            from_addr_port="{}:{}".format(remote_addr, remote_port),
-            response=response,
-            error=error,
         )
 
         if error is None:
@@ -83,7 +90,7 @@ class API:
         else:
             return jsonify({"error": error}), rcode
 
-    # FIXME: THE NEXT THREE API ENDPOINTS ARE FOR TESTING AND SHOULD BE REMOVED / PROPERLY MANAGED BEFORE PRODUCTION!
+    # FIXME: THE NEXT TWO API ENDPOINTS ARE FOR TESTING AND SHOULD BE REMOVED / PROPERLY MANAGED BEFORE PRODUCTION!
     # ToDo: #17-add-api-keys
     def get_appointment(self):
         """
@@ -102,9 +109,16 @@ class API:
             - Unknown appointments are flagged as ``not_found``.
         """
 
+        # Getting the real IP if the server is behind a reverse proxy
+        remote_addr = request.environ.get("HTTP_X_REAL_IP")
+        if not remote_addr:
+            remote_addr = request.environ.get("REMOTE_ADDR")
+
         locator = request.args.get("locator")
         response = []
 
+        logger.info("Received get_appointment request", from_addr="{}".format(remote_addr), locator=locator)
+
         # ToDo: #15-add-system-monitor
         if not isinstance(locator, str) or len(locator) != LOCATOR_LEN_HEX:
             response.append({"locator": locator, "status": "not_found"})
@@ -162,21 +176,6 @@ class API:
 
         return response
 
-    @staticmethod
-    def get_block_count():
-        """
-        Provides the block height of the Watchtower.
-
-        This is a testing endpoint that (most likely) will be removed in production. Its purpose is to give information
-        to testers about the current block so they can define a dummy appointment without having to run a bitcoin node.
-
-        Returns:
-            :obj:`dict`: A json encoded dictionary containing the block height.
-
-        """
-
-        return jsonify({"block_count": BlockProcessor.get_block_count()})
-
     def start(self):
         """
         This function starts the Flask server used to run the API. Adds all the routes to the functions listed above.
@@ -186,7 +185,6 @@ class API:
             "/": (self.add_appointment, ["POST"]),
             "/get_appointment": (self.get_appointment, ["GET"]),
             "/get_all_appointments": (self.get_all_appointments, ["GET"]),
-            "/get_block_count": (self.get_block_count, ["GET"]),
         }
 
         for url, params in routes.items():

pisa/inspector.py

@@ -19,6 +19,10 @@ common.cryptographer.logger = Logger(actor="Cryptographer", log_name_prefix=LOG_
 #        stored + blacklist if multiple wrong requests are received.
 
 
+BLOCKS_IN_A_MONTH = 4320  # 4320 = roughly a month in blocks
+ENCRYPTED_BLOB_MAX_SIZE_HEX = 2 * 2048
+
+
 class Inspector:
     """
     The :class:`Inspector` class is in charge of verifying that the appointment data provided by the user is correct.
@@ -161,7 +165,14 @@ class Inspector:
             if start_time < block_height:
                 message = "start_time is in the past"
             else:
-                message = "start_time is too close to current height"
+                message = (
+                    "start_time is too close to current height. "
+                    "Accepted times are: [current_height+1, current_height+6]"
+                )
+
+        elif start_time > block_height + 6:
+            rcode = errors.APPOINTMENT_FIELD_TOO_BIG
+            message = "start_time is too far in the future. Accepted start times are up to 6 blocks in the future"
 
         if message is not None:
             logger.error(message)
@@ -206,6 +217,10 @@ class Inspector:
             rcode = errors.APPOINTMENT_WRONG_FIELD_TYPE
             message = "wrong end_time data type ({})".format(t)
 
+        elif end_time > block_height + BLOCKS_IN_A_MONTH:  # 4320 = roughly a month in blocks
+            rcode = errors.APPOINTMENT_FIELD_TOO_BIG
+            message = "end_time should be within the next month (<= current_height + 4320)"
+
         elif start_time >= end_time:
             rcode = errors.APPOINTMENT_FIELD_TOO_SMALL
             if start_time > end_time:
@@ -258,6 +273,12 @@ class Inspector:
             rcode = errors.APPOINTMENT_WRONG_FIELD_TYPE
             message = "wrong to_self_delay data type ({})".format(t)
 
+        elif to_self_delay > pow(2, 32):
+            rcode = errors.APPOINTMENT_FIELD_TOO_BIG
+            message = "to_self_delay must fit the transaction nLockTime field ({} > {})".format(
+                to_self_delay, pow(2, 32)
+            )
+
         elif to_self_delay < self.config.get("MIN_TO_SELF_DELAY"):
             rcode = errors.APPOINTMENT_FIELD_TOO_SMALL
             message = "to_self_delay too small. The to_self_delay should be at least {} (current: {})".format(
@@ -301,6 +322,10 @@ class Inspector:
             rcode = errors.APPOINTMENT_WRONG_FIELD_TYPE
             message = "wrong encrypted_blob data type ({})".format(t)
 
+        elif len(encrypted_blob) > ENCRYPTED_BLOB_MAX_SIZE_HEX:
+            rcode = errors.APPOINTMENT_FIELD_TOO_BIG
+            message = "encrypted_blob has to be 2Kib at most (current {})".format(len(encrypted_blob) // 2)
+
         elif re.search(r"^[0-9A-Fa-f]+$", encrypted_blob) is None:
             rcode = errors.APPOINTMENT_WRONG_FIELD_FORMAT
             message = "wrong encrypted_blob format ({})".format(encrypted_blob)

pisa/responder.py

@@ -275,7 +275,6 @@ class Responder:
 
             if len(self.trackers) > 0 and block is not None:
                 txids = block.get("tx")
-                logger.info("List of transactions", txids=txids)
 
                 if self.last_known_block == block.get("previousblockhash"):
                     self.check_confirmations(txids)

pisa/watcher.py

@@ -24,7 +24,7 @@ class Watcher:
 
     The :class:`Watcher` keeps track of the accepted appointments in ``appointments`` and, for new received block,
     checks if any breach has happened by comparing the txids with the appointment locators. If a breach is seen, the
-    :obj:`EncryptedBlob <pisa.encrypted_blob.EncryptedBlob>` of the corresponding appointment is decrypted and the data
+    :obj:`EncryptedBlob <common.encrypted_blob.EncryptedBlob>` of the corresponding appointment is decrypted and the data
     is passed to the :obj:`Responder <pisa.responder.Responder>`.
 
     If an appointment reaches its end with no breach, the data is simply deleted.
@@ -81,7 +81,7 @@ class Watcher:
         the blockchain (``do_watch``) until ``appointments`` is empty.
 
         Once a breach is seen on the blockchain, the :obj:`Watcher` will decrypt the corresponding
-        :obj:`EncryptedBlob <pisa.encrypted_blob.EncryptedBlob>` and pass the information to the
+        :obj:`EncryptedBlob <common.encrypted_blob.EncryptedBlob>` and pass the information to the
         :obj:`Responder <pisa.responder.Responder>`.
 
         The tower may store multiple appointments with the same ``locator`` to avoid DoS attacks based on data
@@ -144,7 +144,6 @@ class Watcher:
 
             if len(self.appointments) > 0 and block is not None:
                 txids = block.get("tx")
-                logger.info("List of transactions", txids=txids)
 
                 expired_appointments = [
                     uuid
@@ -232,7 +231,7 @@ class Watcher:
         """
         Filters what of the found breaches contain valid transaction data.
 
-        The :obj:`Watcher` cannot if a given :obj:`EncryptedBlob <pisa.encrypted_blob.EncryptedBlob>` contains a valid
+        The :obj:`Watcher` cannot if a given :obj:`EncryptedBlob <common.encrypted_blob.EncryptedBlob>` contains a valid
         transaction until a breach if seen. Blobs that contain arbitrary data are dropped and not sent to the
         :obj:`Responder <pisa.responder.Responder>`.
 

test/apps/cli/unit/test_wt_cli.py

@@ -14,7 +14,7 @@ from common.tools import compute_locator
 from common.appointment import Appointment
 from common.cryptographer import Cryptographer
 
-from apps.cli.blob import Blob
+from common.blob import Blob
 import apps.cli.wt_cli as wt_cli
 from test.apps.cli.unit.conftest import get_random_value_hex
 
@@ -38,9 +38,9 @@ dummy_pk_der = dummy_pk.public_bytes(
 # Replace the key in the module with a key we control for the tests
 wt_cli.pisa_public_key = dummy_pk
 # Replace endpoint with dummy one
-wt_cli.pisa_api_server = "dummy.com"
+wt_cli.pisa_api_server = "https://dummy.com"
 wt_cli.pisa_api_port = 12345
-pisa_endpoint = "http://{}:{}/".format(wt_cli.pisa_api_server, wt_cli.pisa_api_port)
+pisa_endpoint = "{}:{}/".format(wt_cli.pisa_api_server, wt_cli.pisa_api_port)
 
 dummy_appointment_request = {
     "tx": get_random_value_hex(192),

test/common/unit/test_appointment.py

@@ -4,7 +4,7 @@ import binascii
 from pytest import fixture
 
 from common.appointment import Appointment
-from pisa.encrypted_blob import EncryptedBlob
+from common.encrypted_blob import EncryptedBlob
 
 from test.pisa.unit.conftest import get_random_value_hex
 

test/common/unit/test_blob.py

@@ -1,6 +1,6 @@
 from binascii import unhexlify
 
-from apps.cli.blob import Blob
+from common.blob import Blob
 from test.pisa.unit.conftest import get_random_value_hex
 
 

test/common/unit/test_cryptographer.py

@@ -5,10 +5,10 @@ from cryptography.hazmat.primitives.asymmetric import ec
 from cryptography.hazmat.primitives import serialization
 
 import common.cryptographer
-from apps.cli.blob import Blob
+from common.blob import Blob
 from common.logger import Logger
 from common.cryptographer import Cryptographer
-from pisa.encrypted_blob import EncryptedBlob
+from common.encrypted_blob import EncryptedBlob
 from test.common.unit.conftest import get_random_value_hex
 
 common.cryptographer.logger = Logger(actor="Cryptographer", log_name_prefix="")

test/common/unit/test_encrypted_blob.py

@@ -1,4 +1,4 @@
-from pisa.encrypted_blob import EncryptedBlob
+from common.encrypted_blob import EncryptedBlob
 from test.pisa.unit.conftest import get_random_value_hex
 
 

test/pisa/e2e/test_basic_e2e.py

@@ -1,13 +1,11 @@
 import json
-import binascii
 from time import sleep
 from riemann.tx import Tx
 
 from pisa import config
 from pisa import HOST, PORT
 from apps.cli import wt_cli
-from apps.cli.blob import Blob
+from common.blob import Blob
-from apps.cli import config as cli_conf
 
 import common.cryptographer
 from common.logger import Logger
@@ -27,7 +25,7 @@ common.cryptographer.logger = Logger(actor="Cryptographer", log_name_prefix="")
 
 # We'll use wt_cli to add appointments. The expected input format is a list of arguments with a json-encoded
 # appointment
-wt_cli.pisa_api_server = HOST
+wt_cli.pisa_api_server = "http://{}".format(HOST)
 wt_cli.pisa_api_port = PORT
 
 # Run pisad

test/pisa/unit/conftest.py

@@ -11,7 +11,7 @@ from cryptography.hazmat.backends import default_backend
 from cryptography.hazmat.primitives.asymmetric import ec
 from cryptography.hazmat.primitives import serialization
 
-from apps.cli.blob import Blob
+from common.blob import Blob
 from pisa.responder import TransactionTracker
 from pisa.tools import bitcoin_cli
 from pisa.db_manager import DBManager