Updated RNS version.

README.md

@@ -1,30 +1,31 @@
-Nomad Network
+Nomad Network - Communicate Freely
 ==========
 
-Communicate Freely.
-
 ![Screenshot](https://github.com/markqvist/NomadNet/raw/master/docs/screenshots/1.png)
 
 The intention with this program is to provide a tool to that allows you to build private and resilient communications platforms that are in complete control and ownership of the people that use them.
 
-Nomad Network is build on [LXMF](https://github.com/markqvist/LXMF) and [Reticulum](https://github.com/markqvist/Reticulum), which together provides the cryptographic mesh functionality and peer-to-peer message routing that Nomad Network relies on. This foundation also makes it possible to use the program over a very wide variety of communication mediums, from packet radio to gigabit fiber.
+Nomad Network is build on [LXMF](https://github.com/markqvist/LXMF) and [Reticulum](https://github.com/markqvist/Reticulum), which together provides the cryptographic mesh functionality and peer-to-peer message routing that Nomad Network relies on. This foundation also makes it possible to use the program over a very wide variety of communication mediums, from packet radio to fiber.
 
 Nomad Network does not need any connections to the public internet to work. In fact, it doesn't even need an IP or Ethernet network. You can use it entirely over packet radio, LoRa or even serial lines. But if you wish, you can bridge islanded Reticulum networks over the Internet or private ethernet networks, or you can build networks running completely over the Internet. The choice is yours.
 
 ## Notable Features
  - Encrypted messaging over packet-radio, LoRa, WiFi or anything else [Reticulum](https://github.com/markqvist/Reticulum) supports.
  - Zero-configuration, minimal-infrastructure mesh communication
+ - Connectable nodes that can host pages and files
+ - Node-side generated pages with PHP, Python, bash or others
+ - Built-in text-based browser for interacting with contents on nodes
+ - An easy to use and bandwidth efficient markup language for writing pages
+ - Page caching in browser
 
 ## Current Status
+The current version of the program should be considered a beta release. The program works well, but there will most probably be bugs and possibly sub-optimal performance in some scenarios. On the other hand, this is the ideal time to have an influence on the direction of the development of Nomad Network. To do so, join the discussion, report bugs and request features here on the GitHub project.
 
-
-The current version of the program should be considered an alpha release. The program works well, but there will most probably be bugs and possibly sub-optimal performance in some scenarios. On the other hand, this is the ideal time to have an influence on the direction of the development of Nomad Network. To do so, join the discussion, report bugs and request features here on the GitHub project.
-
-Development is ongoing and current features being implemented are:
-
- - Propagated messaging and discussion threads
- - Connectable nodes that can host pages, files and other resources
- - Collaborative information sharing and spatial map-style "wikis"
+### Feature roadmap
+ - Access control and authentication for nodes, pages and files
+ - Network-wide propagated messaging and discussion threads
+ - Geospatial information sharing
+ - Distributed Marketplace
 
 ## Dependencies:
  - Python 3
@@ -42,7 +43,21 @@ pip3 install nomadnet
 nomadnet
 ```
 
+The first time the program is running, you will be presented with the guide section, which contains all the information you need to start using Nomad Network.
+
 To use Nomad Network on packet radio or LoRa, you will need to configure your Reticulum installation to use any relevant packet radio TNCs or LoRa devices on your system. See the Reticulum documentation for info.
 
 ## Caveat Emptor
-Nomad Network is experimental software, and should be considered as such. While it has been built with cryptography best-practices very foremost in mind, it _has not_ been externally security audited, and there could very well be privacy-breaking bugs. If you want to help out, or help sponsor an audit, please do get in touch.
+Nomad Network is beta software, and should be considered as such. While it has been built with cryptography best-practices very foremost in mind, it _has not_ been externally security audited, and there could very well be privacy-breaking bugs. If you want to help out, or help sponsor an audit, please do get in touch.
+
+## Screenshots
+
+![Screenshot 1](https://github.com/markqvist/NomadNet/raw/master/docs/screenshots/1.png)
+
+![Screenshot 2](https://github.com/markqvist/NomadNet/raw/master/docs/screenshots/2.png)
+
+![Screenshot 3](https://github.com/markqvist/NomadNet/raw/master/docs/screenshots/3.png)
+
+![Screenshot 4](https://github.com/markqvist/NomadNet/raw/master/docs/screenshots/4.png)
+
+![Screenshot 5](https://github.com/markqvist/NomadNet/raw/master/docs/screenshots/5.png)

nomadnet/Conversation.py

@@ -7,6 +7,7 @@ from nomadnet.Directory import DirectoryEntry
 
 class Conversation:
     cached_conversations = {}
+    unread_conversations = {}
     created_callback = None
 
     aspect_filter = "lxmf.delivery"
@@ -57,6 +58,16 @@ class Conversation:
             conversation = Conversation.cached_conversations[RNS.hexrep(source_hash, delimit=False)]
             conversation.scan_storage()
 
+        if not source_hash in Conversation.unread_conversations:
+            Conversation.unread_conversations[source_hash] = True
+            try:
+                dirname = RNS.hexrep(source_hash, delimit=False)
+                open(app.conversationpath + "/" + dirname + "/unread", 'a').close()
+            except Exception as e:
+                pass
+
+            Conversation.created_callback()
+
         return ingested_path
 
     @staticmethod
@@ -70,6 +81,13 @@ class Conversation:
                     app_data         = RNS.Identity.recall_app_data(source_hash)
                     display_name     = app.directory.display_name(source_hash)
 
+                    unread = False
+                    if source_hash in Conversation.unread_conversations:
+                        unread = True
+                    elif os.path.isfile(app.conversationpath + "/" + dirname + "/unread"):
+                        Conversation.unread_conversations[source_hash] = True
+                        unread = True
+
                     if display_name == None and app_data:
                         display_name = app_data.decode("utf-8")
 
@@ -80,7 +98,7 @@ class Conversation:
                     
                     trust_level      = app.directory.trust_level(source_hash, display_name)
                     
-                    entry = (source_hash_text, display_name, trust_level, sort_name)
+                    entry = (source_hash_text, display_name, trust_level, sort_name, unread)
                     conversations.append(entry)
 
                 except Exception as e:
@@ -114,6 +132,7 @@ class Conversation:
         self.source_known       = False
         self.source_trusted     = False
         self.source_blocked     = False
+        self.unread             = False
 
         self.__changed_callback = None
 
@@ -143,6 +162,11 @@ class Conversation:
                 message_path = self.messages_path + "/" + filename
                 self.messages.append(ConversationMessage(message_path))
 
+        new_len = len(self.messages)
+
+        if new_len > old_len:
+            self.unread = True
+
         if self.__changed_callback != None:
             self.__changed_callback(self)
 
@@ -156,6 +180,15 @@ class Conversation:
         for purged_message in purged_messages:
             self.messages.remove(purged_message)
 
+    def clear_history(self):
+        purged_messages = []
+        for conversation_message in self.messages:
+            purged_messages.append(conversation_message)
+            conversation_message.purge()
+
+        for purged_message in purged_messages:
+            self.messages.remove(purged_message)
+
     def register_changed_callback(self, callback):
         self.__changed_callback = callback
 

nomadnet/Node.py

@@ -2,6 +2,7 @@ import os
 import RNS
 import time
 import threading
+import subprocess
 import RNS.vendor.umsgpack as msgpack
 
 class Node:
@@ -44,7 +45,6 @@ class Node:
 
         for page in self.servedpages:
             request_path = "/page"+page.replace(self.app.pagespath, "")
-
             self.destination.register_request_handler(
                 request_path,
                 response_generator = self.serve_page,
@@ -55,6 +55,14 @@ class Node:
         self.servedfiles = []
         self.scan_files(self.app.filespath)
 
+        for file in self.servedfiles:
+            request_path = "/file"+file.replace(self.app.filespath, "")
+            self.destination.register_request_handler(
+                request_path,
+                response_generator = self.serve_file,
+                allow = RNS.Destination.ALLOW_ALL
+            )
+
     def scan_pages(self, base_path):
         files = [file for file in os.listdir(base_path) if os.path.isfile(os.path.join(base_path, file)) and file[:1] != "."]
         directories = [file for file in os.listdir(base_path) if os.path.isdir(os.path.join(base_path, file)) and file[:1] != "."]
@@ -76,20 +84,40 @@ class Node:
             self.scan_files(base_path+"/"+directory)
 
     def serve_page(self, path, data, request_id, remote_identity, requested_at):
-        RNS.log("Request "+RNS.prettyhexrep(request_id)+" for: "+str(path), RNS.LOG_VERBOSE)
+        RNS.log("Page request "+RNS.prettyhexrep(request_id)+" for: "+str(path), RNS.LOG_VERBOSE)
         file_path = path.replace("/page", self.app.pagespath, 1)
         try:
-            RNS.log("Serving file: "+file_path, RNS.LOG_VERBOSE)
-            fh = open(file_path, "rb")
-            response_data = fh.read()
-            fh.close()
-            return response_data
+            RNS.log("Serving page: "+file_path, RNS.LOG_VERBOSE)
+            if os.access(file_path, os.X_OK):
+                generated = subprocess.run([file_path], stdout=subprocess.PIPE)
+                return generated.stdout
+            else:
+                fh = open(file_path, "rb")
+                response_data = fh.read()
+                fh.close()
+                return response_data
 
         except Exception as e:
             RNS.log("Error occurred while handling request "+RNS.prettyhexrep(request_id)+" for: "+str(path), RNS.LOG_ERROR)
             RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
             return None
 
+    # TODO: Improve file handling, this will be slow for large files
+    def serve_file(self, path, data, request_id, remote_identity, requested_at):
+        RNS.log("File request "+RNS.prettyhexrep(request_id)+" for: "+str(path), RNS.LOG_VERBOSE)
+        file_path = path.replace("/file", self.app.filespath, 1)
+        file_name = path.replace("/file/", "", 1)
+        try:
+            RNS.log("Serving file: "+file_path, RNS.LOG_VERBOSE)
+            fh = open(file_path, "rb")
+            file_data = fh.read()
+            fh.close()
+            return [file_name, file_data]
+
+        except Exception as e:
+            RNS.log("Error occurred while handling request "+RNS.prettyhexrep(request_id)+" for: "+str(path), RNS.LOG_ERROR)
+            RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
+            return None
 
     def serve_default_index(self, path, data, request_id, remote_identity, requested_at):
         RNS.log("Serving default index for request "+RNS.prettyhexrep(request_id)+" for: "+str(path), RNS.LOG_VERBOSE)
@@ -98,6 +126,7 @@ class Node:
     def announce(self):
         self.app_data = self.name.encode("utf-8")
         self.last_announce = time.time()
+        self.app.peer_settings["node_last_announce"] = self.last_announce
         self.destination.announce(app_data=self.app_data)
 
     def __jobs(self):

nomadnet/NomadNetworkApp.py

@@ -53,6 +53,13 @@ class NomadNetworkApp:
 
         self.pagespath         = self.configdir+"/storage/pages"
         self.filespath         = self.configdir+"/storage/files"
+        self.cachepath         = self.configdir+"/storage/cache"
+
+        self.downloads_path    = os.path.expanduser("~/Downloads")
+
+        self.firstrun          = False
+
+        self.peer_announce_at_start = True
 
         if not os.path.isdir(self.storagepath):
             os.makedirs(self.storagepath)
@@ -72,6 +79,9 @@ class NomadNetworkApp:
         if not os.path.isdir(self.filespath):
             os.makedirs(self.filespath)
 
+        if not os.path.isdir(self.cachepath):
+            os.makedirs(self.cachepath)
+
         if os.path.isfile(self.configpath):
             try:
                 self.config = ConfigObj(self.configpath)
@@ -89,7 +99,7 @@ class NomadNetworkApp:
         else:
             RNS.log("Could not load config file, creating default configuration file...")
             self.createDefaultConfig()
-
+            self.firstrun = True
 
         if os.path.isfile(self.identitypath):
             try:
@@ -119,6 +129,10 @@ class NomadNetworkApp:
                 file = open(self.peersettingspath, "rb")
                 self.peer_settings = msgpack.unpackb(file.read())
                 file.close()
+
+                if not "node_last_announce" in self.peer_settings:
+                    self.peer_settings["node_last_announce"] = None
+
             except Exception as e:
                 RNS.log("Could not load local peer settings from "+self.peersettingspath, RNS.LOG_ERROR)
                 RNS.log("The contained exception was: %s" % (str(e)), RNS.LOG_ERROR)
@@ -127,9 +141,10 @@ class NomadNetworkApp:
             try:
                 RNS.log("No peer settings file found, creating new...")
                 self.peer_settings = {
-                    "display_name": "",
+                    "display_name": "Anonymous Peer",
                     "announce_interval": None,
                     "last_announce": None,
+                    "node_last_announce": None,
                 }
                 self.save_peer_settings()
                 RNS.log("Created new peer settings file")
@@ -166,6 +181,9 @@ class NomadNetworkApp:
         RNS.Transport.register_announce_handler(nomadnet.Conversation)
         RNS.Transport.register_announce_handler(nomadnet.Directory)
 
+        if self.peer_announce_at_start:
+            self.announce_now()
+
         nomadnet.ui.spawn(self.uimode)
 
     def set_display_name(self, display_name):
@@ -205,6 +223,18 @@ class NomadNetworkApp:
     def conversations(self):
         return nomadnet.Conversation.conversation_list(self)
 
+    def conversation_is_unread(self, source_hash):
+        if bytes.fromhex(source_hash) in nomadnet.Conversation.unread_conversations:
+            return True
+        else:
+            return False
+
+    def mark_conversation_read(self, source_hash):
+        if bytes.fromhex(source_hash) in nomadnet.Conversation.unread_conversations:
+            nomadnet.Conversation.unread_conversations.pop(bytes.fromhex(source_hash))
+            if os.path.isfile(self.conversationpath + "/" + source_hash + "/unread"):
+                os.unlink(self.conversationpath + "/" + source_hash + "/unread")
+
     def createDefaultConfig(self):
         self.config = ConfigObj(__default_nomadnet_config__)
         self.config.filename = self.configpath
@@ -242,6 +272,14 @@ class NomadNetworkApp:
                     value = self.config["client"].as_bool(option)
                     self.enable_client = value
 
+                if option == "downloads_path":
+                    value = self.config["client"]["downloads_path"]
+                    self.downloads_path = os.path.expanduser(value)
+
+                if option == "announce_at_start":
+                    value = self.config["client"].as_bool(option)
+                    self.peer_announce_at_start = value
+
                 if option == "user_interface":
                     value = value.lower()
                     if value == "none":
@@ -327,7 +365,8 @@ class NomadNetworkApp:
             if not "announce_at_start" in self.config["node"]:
                 self.node_announce_at_start = False
             else:
-                self.node_announce_at_start = self.config["node"]["announce_at_start"]
+                value = self.config["node"].as_bool("announce_at_start")
+                self.node_announce_at_start = value
 
             if not "announce_interval" in self.config["node"]:
                 self.node_announce_interval = 720
@@ -379,25 +418,34 @@ destination = file
 
 enable_client = yes
 user_interface = text
+downloads_path = ~/Downloads
+
+# By default, the peer is announced at startup
+# to let other peers reach it immediately.
+announce_at_start = yes
 
 [textui]
 
 # Amount of time to show intro screen
 intro_time = 1
 
+# You can specify the display theme.
+# theme = dark
+theme = light
+
 # Specify the number of colors to use
 # valid colormodes are:
 # monochrome, 16, 88, 256 and 24bit
 #
-# The default is a conservative 16 colors,
-# but 256 colors can probably be used on
-# most terminals. Some terminals support
+# The default is a conservative 256 colors.
+# If your terminal does not support this,
+# you can lower it. Some terminals support
 # 24 bit color.
 
 # colormode = monochrome
-colormode = 16
+# colormode = 16
 # colormode = 88
-# colormode = 256
+colormode = 256
 # colormode = 24bit
 
 # By default, unicode glyphs are used. If

nomadnet/_version.py

@@ -1 +1 @@
-__version__ = "0.0.7"
+__version__ = "0.0.9"

nomadnet/ui/TextUI.py

@@ -1,6 +1,7 @@
 import RNS
-import importlib
+import urwid
 import time
+import os
 
 import nomadnet
 from nomadnet.ui.textui import *
@@ -18,22 +19,22 @@ THEMES = {
     THEME_DARK: {
         "urwid_theme": [
             # Style name                    # 16-color style                        # Monochrome style          # 88, 256 and true-color style
-            ('heading',                     'light gray,underline', 'default',      'underline',                'g93,underline', 'default'),
-            ('menubar',                     'black', 'light gray',                  'standout',                 '#111', '#bbb'),
-            ('scrollbar',                   'light gray', 'default',                'standout',                 '#444', 'default'),
-            ('shortcutbar',                 'black', 'light gray',                  'standout',                 '#111', '#bbb'),
-            ('body_text',                   'white', 'default',                     'default',                  '#ddd', 'default'),
-            ('error_text',                  'dark red', 'default',                  'default',                  'dark red', 'default'),
-            ('warning_text',                'yellow', 'default',                    'default',                  '#ba4', 'default'),
-            ('inactive_text',               'dark gray', 'default',                 'default',                  'dark gray', 'default'),
-            ('buttons',                     'light green,bold', 'default',          'default',                  '#00a533', 'default'),
-            ('msg_editor',                  'black', 'light cyan',                  'standout',                 '#111', '#0bb'),
-            ("msg_header_ok",               'black', 'light green',                 'standout',                 '#111', '#6b2'),
-            ("msg_header_caution",          'black', 'yellow',                      'standout',                 '#111', '#fd3'),
-            ("msg_header_sent",             'black', 'light gray',                  'standout',                 '#111', '#ddd'),
-            ("msg_header_delivered",        'black', 'light blue',                  'standout',                 '#111', '#28b'),
-            ("msg_header_failed",           'black', 'dark gray',                   'standout',                 '#000', "#777"),
-            ("msg_warning_untrusted",       'black', 'dark red',                    'standout',                 '#111', 'dark red'),
+            ("heading",                     "light gray,underline", "default",      "underline",                "g93,underline", "default"),
+            ("menubar",                     "black", "light gray",                  "standout",                 "#111", "#bbb"),
+            ("scrollbar",                   "light gray", "default",                "standout",                 "#444", "default"),
+            ("shortcutbar",                 "black", "light gray",                  "standout",                 "#111", "#bbb"),
+            ("body_text",                   "white", "default",                     "default",                  "#ddd", "default"),
+            ("error_text",                  "dark red", "default",                  "default",                  "dark red", "default"),
+            ("warning_text",                "yellow", "default",                    "default",                  "#ba4", "default"),
+            ("inactive_text",               "dark gray", "default",                 "default",                  "dark gray", "default"),
+            ("buttons",                     "light green,bold", "default",          "default",                  "#00a533", "default"),
+            ("msg_editor",                  "black", "light cyan",                  "standout",                 "#111", "#0bb"),
+            ("msg_header_ok",               "black", "light green",                 "standout",                 "#111", "#6b2"),
+            ("msg_header_caution",          "black", "yellow",                      "standout",                 "#111", "#fd3"),
+            ("msg_header_sent",             "black", "light gray",                  "standout",                 "#111", "#ddd"),
+            ("msg_header_delivered",        "black", "light blue",                  "standout",                 "#111", "#28b"),
+            ("msg_header_failed",           "black", "dark gray",                   "standout",                 "#000", "#777"),
+            ("msg_warning_untrusted",       "black", "dark red",                    "standout",                 "#111", "dark red"),
             ("list_focus",                  "black", "light gray",                  "standout",                 "#111", "#aaa"),
             ("list_off_focus",              "black", "dark gray",                   "standout",                 "#111", "#777"),
             ("list_trusted",                "dark green", "default",                "default",                  "#6b2", "default"),
@@ -43,10 +44,43 @@ THEMES = {
             ("list_untrusted",              "dark red", "default",                  "default",                  "#a22", "default"),
             ("list_focus_untrusted",        "black", "light gray",                  "standout",                 "#810", "#aaa"),
             ("topic_list_normal",           "white", "default",                     "default",                  "#ddd", "default"),
-            ('browser_controls',            "light gray", 'default',                     'default',                  '#bbb', 'default'),
+            ("browser_controls",            "light gray", "default",                "default",                  "#bbb", "default"),
             ("progress_full",               "black", "light gray",                  "standout",                 "#111", "#bbb"),
             ("progress_empty",              "light gray", "default",                "default",                  "#ddd", "default"),
         ],        
+    },
+    THEME_LIGHT: {
+        "urwid_theme": [
+            # Style name                    # 16-color style                        # Monochrome style          # 88, 256 and true-color style
+            ("heading",                     "dark gray,underline", "default",      "underline",                "g93,underline", "default"),
+            ("menubar",                     "black", "dark gray",                  "standout",                 "#111", "#bbb"),
+            ("scrollbar",                   "dark gray", "default",                "standout",                 "#444", "default"),
+            ("shortcutbar",                 "black", "dark gray",                  "standout",                 "#111", "#bbb"),
+            ("body_text",                   "black", "default",                    "default",                  "#222", "default"),
+            ("error_text",                  "dark red", "default",                 "default",                  "dark red", "default"),
+            ("warning_text",                "yellow", "default",                   "default",                  "#ba4", "default"),
+            ("inactive_text",               "light gray", "default",               "default",                  "dark gray", "default"),
+            ("buttons",                     "light green,bold", "default",         "default",                  "#00a533", "default"),
+            ("msg_editor",                  "black", "dark cyan",                  "standout",                 "#111", "#0bb"),
+            ("msg_header_ok",               "black", "dark green",                 "standout",                 "#111", "#6b2"),
+            ("msg_header_caution",          "black", "yellow",                     "standout",                 "#111", "#fd3"),
+            ("msg_header_sent",             "black", "dark gray",                  "standout",                 "#111", "#ddd"),
+            ("msg_header_delivered",        "black", "light blue",                 "standout",                 "#111", "#28b"),
+            ("msg_header_failed",           "black", "dark gray",                  "standout",                 "#000", "#777"),
+            ("msg_warning_untrusted",       "black", "dark red",                   "standout",                 "#111", "dark red"),
+            ("list_focus",                  "black", "dark gray",                  "standout",                 "#111", "#aaa"),
+            ("list_off_focus",              "black", "dark gray",                  "standout",                 "#111", "#777"),
+            ("list_trusted",                "dark green", "default",               "default",                  "#4a0", "default"),
+            ("list_focus_trusted",          "black", "dark gray",                  "standout",                 "#150", "#aaa"),
+            ("list_unknown",                "dark gray", "default",                "default",                  "#444", "default"),
+            ("list_normal",                 "dark gray", "default",                "default",                  "#444", "default"),
+            ("list_untrusted",              "dark red", "default",                 "default",                  "#a22", "default"),
+            ("list_focus_untrusted",        "black", "dark gray",                  "standout",                 "#810", "#aaa"),
+            ("topic_list_normal",           "black", "default",                    "default",                  "#222", "default"),
+            ("browser_controls",            "dark gray", "default",                "default",                  "#444", "default"),
+            ("progress_full",               "black", "dark gray",                  "standout",                 "#111", "#bbb"),
+            ("progress_empty",              "dark gray", "default",                "default",                  "#ddd", "default"),
+        ],        
     }
 }
 
@@ -61,20 +95,22 @@ GLYPHS = {
     ("check",           "=",         "\u2713",      "\u2713"),
     ("cross",           "X",         "\u2715",      "\u2715"),
     ("unknown",         "?",         "?",           "?"),
-    ("lock",            "E",         "\U0001f512",  "\uf023"),
-    ("unlock",          "!",         "\U0001f513",  "\uf09c"),
+    ("encrypted",       "",          "\u26BF",      "\uf023"),
+    ("plaintext",       "!",         "!",           "\uf06e "),
     ("arrow_r",         "->",        "\u2192",      "\u2192"),
     ("arrow_l",         "<-",        "\u2190",      "\u2190"),
     ("arrow_u",         "/\\",       "\u2191",      "\u2191"),
     ("arrow_d",         "\\/",       "\u2193",      "\u2193"),
     ("warning",         "!",         "\u26a0",      "\uf12a"),
     ("info",            "i",         "\u2139",      "\ufb4d"),
+    ("unread",          "[U]",       "\u2709",      "\uf003 "),
     ("divider1",        "-",         "\u2504",      "\u2504"),
-    ("peer",            "P",         "\U0001F464",  "\uf415"),
-    ("node",            "N",         "\U0001F5A5",  "\uf502"),
-    ("page",            "",          "\U0001F5B9",  "\uf719 "),
-    ("speed",           "",          "\U0001F5B9",  "\uf9c4"),
+    ("peer",            "[P]",       "\u24c5 ",     "\uf415"),
+    ("node",            "[N]",       "\u24c3 ",     "\uf502"),
+    ("page",            "",          "\u25a4",      "\uf719 "),
+    ("speed",           "",          "\u25F7",      "\uf9c4"),
     ("decoration_menu", "",          "",            " \uf93a"),
+    ("globe",           "",          "",            "\uf484"),
 }
 
 class TextUI:
@@ -84,12 +120,13 @@ class TextUI:
         self.app.ui = self
         self.loop = None
 
-        if importlib.util.find_spec("urwid") != None:
-            import urwid
-        else:
-            RNS.log("The text-mode user interface requires Urwid to be installed on your system.", RNS.LOG_ERROR)
-            RNS.log("You can install it with the command: pip3 install urwid", RNS.LOG_ERROR)
-            nomadnet.panic()
+        # TODO: Remove
+        # if importlib.util.find_spec("urwid") != None:
+        #     import urwid
+        # else:
+        #     RNS.log("The text-mode user interface requires Urwid to be installed on your system.", RNS.LOG_ERROR)
+        #     RNS.log("You can install it with the command: pip3 install urwid", RNS.LOG_ERROR)
+        #     nomadnet.panic()
 
         urwid.set_encoding("UTF-8")
 
@@ -129,11 +166,21 @@ class TextUI:
         if intro_timeout > 0:
             self.loop.set_alarm_in(intro_timeout, self.display_main)
 
-        # TODO: Probably remove this at some point when better terminal
-        # color capability detection has been implemented
-        if colormode > 16:
-            RNS.log("Starting Text UI in "+str(colormode)+" color mode. If no UI appears, try adjusting your color settings in "+str(self.app.configdir)+"/config", RNS.LOG_INFO, _override_destination = True)
-        
+        if "KONSOLE_VERSION" in os.environ:
+            if colormode > 16:
+                RNS.log("", RNS.LOG_WARNING, _override_destination = True)
+                RNS.log("", RNS.LOG_WARNING, _override_destination = True)
+                RNS.log("You are using the terminal emulator Konsole.", RNS.LOG_WARNING, _override_destination = True)
+                RNS.log("If you are not seeing the user interface, it is due to a bug in Konsole/urwid.", RNS.LOG_WARNING, _override_destination = True)
+                RNS.log("", RNS.LOG_WARNING, _override_destination = True)
+
+                RNS.log("To circumvent this, use another terminal emulator, or launch nomadnet within a", RNS.LOG_WARNING, _override_destination = True)
+                RNS.log("screen session, using a command like the following:", RNS.LOG_WARNING, _override_destination = True)
+                RNS.log("", RNS.LOG_WARNING, _override_destination = True)
+                RNS.log("screen nomadnet", RNS.LOG_WARNING, _override_destination = True)
+                RNS.log("", RNS.LOG_WARNING, _override_destination = True)
+                RNS.log("Press ctrl-c to exit now and try again.", RNS.LOG_WARNING, _override_destination = True)
+
         self.set_colormode(colormode)
 
         self.loop.run()

nomadnet/ui/textui/Browser.py

@@ -1,18 +1,25 @@
 import RNS
+import os
 import time
 import urwid
 import nomadnet
+import subprocess
 import threading
 from .MicronParser import markup_to_attrmaps
 from nomadnet.vendor.Scrollable import *
 
-# TODO: REMOVE
-import os
-
 class BrowserFrame(urwid.Frame):
     def keypress(self, size, key):
         if key == "ctrl w":
             self.delegate.disconnect()
+        elif key == "ctrl d":
+            self.delegate.back()
+        elif key == "ctrl f":
+            self.delegate.forward()
+        elif key == "ctrl r":
+            self.delegate.reload()
+        elif key == "ctrl u":
+            self.delegate.url_dialog()
         elif self.get_focus() == "body":
             return super(BrowserFrame, self).keypress(size, key)
             # if key == "up" and self.delegate.messagelist.top_is_visible:
@@ -26,19 +33,21 @@ class BrowserFrame(urwid.Frame):
 
 class Browser:
     DEFAULT_PATH       = "/page/index.mu"
-    DEFAULT_TIMEOUT    = 5
+    DEFAULT_TIMEOUT    = 10
+    DEFAULT_CACHE_TIME = 12*60*60
 
     NO_PATH            = 0x00
     PATH_REQUESTED     = 0x01
     ESTABLISHING_LINK  = 0x02
-    LINK_ESTABLISHED   = 0x03
-    REQUESTING         = 0x04
-    REQUEST_SENT       = 0x05
-    REQUEST_FAILED     = 0x06
-    REQUEST_TIMEOUT    = 0x07
-    RECEIVING_RESPONSE = 0x08
-    DONE               = 0xFF
+    LINK_TIMEOUT       = 0x03
+    LINK_ESTABLISHED   = 0x04
+    REQUESTING         = 0x05
+    REQUEST_SENT       = 0x06
+    REQUEST_FAILED     = 0x07
+    REQUEST_TIMEOUT    = 0x08
+    RECEIVING_RESPONSE = 0x09
     DISCONECTED        = 0xFE
+    DONE               = 0xFF
 
     def __init__(self, app, app_name, aspects, destination_hash = None, path = None, auth_identity = None, delegate = None):
         self.app = app
@@ -52,18 +61,29 @@ class Browser:
         self.last_keypress = None
 
         self.link = None
+        self.loopback = None
         self.status = Browser.DISCONECTED
         self.response_progress = 0
         self.response_size = None
         self.response_transfer_size = None
+        self.saved_file_name = None
         self.page_data = None
         self.displayed_page_data = None
         self.auth_identity = auth_identity
         self.display_widget = None
+        self.link_status_showing = False
+        self.link_target = None
         self.frame = None
         self.attr_maps = []
         self.build_display()
 
+        self.history = []
+        self.history_ptr = 0
+        self.history_inc = False
+        self.history_dec = False
+        self.reloading = False
+        self.loaded_from_cache = False
+
         if self.path == None:
             self.path = Browser.DEFAULT_PATH
 
@@ -80,14 +100,44 @@ class Browser:
                 path = self.path
             return RNS.hexrep(self.destination_hash, delimit=False)+":"+path
 
-    def handle_link(self, link_target):
-        RNS.log("Browser handling link to: "+str(link_target), RNS.LOG_DEBUG)
-        try:
-            self.retrieve_url(link_target)
-        except Exception as e:
-            self.browser_footer = urwid.Text("Could not open link: "+str(e))
+    def url_hash(self, url):
+        if url == None:
+            return None
+        else:
+            url = url.encode("utf-8")
+            return RNS.hexrep(RNS.Identity.full_hash(url), delimit=False)
+
+
+    def marked_link(self, link_target):
+        if self.status == Browser.DONE:
+            self.link_target = link_target
+            self.app.ui.loop.set_alarm_in(0.1, self.marked_link_job)
+
+    def marked_link_job(self, sender, event):
+        link_target = self.link_target
+
+        if link_target == None:
+            if self.link_status_showing:
+                self.browser_footer = self.make_status_widget()
+                self.frame.contents["footer"] = (self.browser_footer, self.frame.options())
+                self.link_status_showing = False
+        else:
+            self.link_status_showing = True
+            self.browser_footer = urwid.AttrMap(urwid.Pile([urwid.Divider(self.g["divider1"]), urwid.Text("Link to: "+str(link_target))]), "browser_controls")
             self.frame.contents["footer"] = (self.browser_footer, self.frame.options())
 
+    def handle_link(self, link_target):
+        if self.status >= Browser.DISCONECTED:
+            RNS.log("Browser handling link to: "+str(link_target), RNS.LOG_DEBUG)
+            try:
+                self.retrieve_url(link_target)
+            except Exception as e:
+                self.browser_footer = urwid.Text("Could not open link: "+str(e))
+                self.frame.contents["footer"] = (self.browser_footer, self.frame.options())
+        else:
+            RNS.log("Browser aleady hadling link, cannot handle link to: "+str(link_target), RNS.LOG_DEBUG)
+
+
 
     def micron_released_focus(self):
         if self.delegate != None:
@@ -101,7 +151,8 @@ class Browser:
 
         self.frame = BrowserFrame(self.browser_body, header=self.browser_header, footer=self.browser_footer)
         self.frame.delegate = self
-        self.display_widget = urwid.AttrMap(urwid.LineBox(self.frame, title="Remote Node"), "inactive_text")
+        self.linebox = urwid.LineBox(self.frame, title="Remote Node")
+        self.display_widget = urwid.AttrMap(self.linebox, "inactive_text")
 
     def make_status_widget(self):
         if self.response_progress > 0:
@@ -146,14 +197,24 @@ class Browser:
             self.browser_body = urwid.Filler(urwid.Text("Disconnected\n"+self.g["arrow_l"]+"  "+self.g["arrow_r"], align="center"), "middle")
             self.browser_footer = urwid.Text("")
             self.browser_header = urwid.Text("")
+            self.linebox.set_title("Remote Node")
         else:
             self.display_widget.set_attr_map({None: "body_text"})
             self.browser_header = self.make_control_widget()
+            remote_display_string = self.app.directory.simplest_display_str(self.destination_hash)
+            if remote_display_string == RNS.prettyhexrep(self.loopback):
+                remote_display_string = self.app.node.name
+
+            self.linebox.set_title(remote_display_string)
 
             if self.status == Browser.DONE:
                 self.browser_footer = self.make_status_widget()
                 self.update_page_display()
             
+            elif self.status == Browser.LINK_TIMEOUT:
+                self.browser_body = self.make_request_failed_widget()
+                self.browser_footer = urwid.Text("")
+            
             elif self.status <= Browser.REQUEST_SENT:
                 if len(self.attr_maps) == 0:
                     self.browser_body = urwid.Filler(urwid.Text("Retrieving\n["+self.current_url()+"]", align="center"), "middle")
@@ -177,7 +238,7 @@ class Browser:
 
     def update_page_display(self):
         pile = urwid.Pile(self.attr_maps)
-        self.browser_body = urwid.AttrMap(ScrollBar(Scrollable(pile), thumb_char="\u2503", trough_char=" "), "scrollbar")
+        self.browser_body = urwid.AttrMap(ScrollBar(Scrollable(pile, force_forward_keypress=True), thumb_char="\u2503", trough_char=" "), "scrollbar")
 
     def identify(self):
         if self.link != None:
@@ -195,6 +256,11 @@ class Browser:
         self.response_size = None
         self.response_transfer_size = None
 
+        self.history = []
+        self.history_ptr = 0
+        self.history_inc = False
+        self.history_dec = False
+
         self.update_display()
 
 
@@ -239,9 +305,19 @@ class Browser:
             raise ValueError("Malformed URL")
 
         if destination_hash != None and path != None:
-            self.set_destination_hash(destination_hash)
-            self.set_path(path)
-            self.load_page()
+            if path.startswith("/file/"):
+                if destination_hash != self.loopback:
+                    if destination_hash == self.destination_hash:
+                        self.download_file(destination_hash, path)
+                    else:
+                        RNS.log("Cannot request file download from a node that is not currently connected.", RNS.LOG_ERROR)
+                        RNS.log("The requested URL was: "+str(url), RNS.LOG_ERROR)
+                else:
+                    self.download_local_file(path)
+            else:
+                self.set_destination_hash(destination_hash)
+                self.set_path(path)
+                self.load_page()
 
     def set_destination_hash(self, destination_hash):
         if len(destination_hash) == RNS.Identity.TRUNCATED_HASHLENGTH//8:
@@ -258,18 +334,196 @@ class Browser:
     def set_timeout(self, timeout):
         self.timeout = timeout
 
+    def download_local_file(self, path):
+        try:
+            file_path = self.app.filespath+path.replace("/file", "", 1)
+            if os.path.isfile(file_path):
+                file_name = os.path.basename(file_path)
+                file_destination = self.app.downloads_path+"/"+file_name
+
+                counter = 0
+                while os.path.isfile(file_destination):
+                    counter += 1
+                    file_destination = self.app.downloads_path+"/"+file_name+"."+str(counter)
+
+                fs = open(file_path, "rb")
+                fd = open(file_destination, "wb")
+                fd.write(fs.read())
+                fd.close()
+                fs.close()
+
+                self.saved_file_name = file_destination.replace(self.app.downloads_path+"/", "", 1)
+                
+                self.update_display()
+            else:
+                RNS.log("The requested local download file does not exist: "+str(file_path), RNS.LOG_ERROR)
+
+            self.status = Browser.DONE
+            self.response_progress = 0
+            
+        except Exception as e:
+            RNS.log("An error occurred while handling file response. The contained exception was: "+str(e), RNS.LOG_ERROR)
+
+    def download_file(self, destination_hash, path):
+        if self.link != None and self.link.destination.hash == self.destination_hash:
+            # Send the request
+            self.status = Browser.REQUESTING
+            self.response_progress = 0
+            self.response_size = None
+            self.response_transfer_size = None
+            self.saved_file_name = None
+
+            self.update_display()
+            receipt = self.link.request(
+                path,
+                data = None,
+                response_callback = self.file_received,
+                failed_callback = self.request_failed,
+                progress_callback = self.response_progressed
+            )
+
+            if receipt:
+                self.last_request_receipt = receipt
+                self.last_request_id = receipt.request_id
+                self.status = Browser.REQUEST_SENT
+                self.update_display()
+            else:
+                self.link.teardown()
+
+    def write_history(self):
+        entry = [self.destination_hash, self.path]
+        self.history.insert(self.history_ptr, entry)
+        self.history_ptr += 1
+
+        if len(self.history) > self.history_ptr:
+            self.history = self.history[:self.history_ptr]
+
+    def back(self):
+        target_ptr = self.history_ptr-1
+        if not self.history_inc and not self.history_dec:
+            if target_ptr > 0:
+                self.history_dec = True
+                entry = self.history[target_ptr-1]
+                url = RNS.hexrep(entry[0], delimit=False)+":"+entry[1]
+                self.history_ptr = target_ptr
+                self.retrieve_url(url)
+
+    def forward(self):
+        target_ptr = self.history_ptr+1
+        if not self.history_inc and not self.history_dec:
+            if target_ptr <= len(self.history):
+                self.history_dec = True
+                entry = self.history[target_ptr-1]
+                url = RNS.hexrep(entry[0], delimit=False)+":"+entry[1]
+                self.history_ptr = target_ptr
+                self.retrieve_url(url)
+
+    def reload(self):
+        if not self.reloading and self.status == Browser.DONE:
+            self.reloading = True
+            self.uncache_page(self.current_url())
+            self.load_page()
+
+    def close_dialogs(self):
+        options = self.delegate.columns.options("weight", self.delegate.right_area_width)
+        self.delegate.columns.contents[1] = (self.display_widget, options)
+
+    def url_dialog(self):
+        e_url = UrlEdit(caption="URL : ", edit_text=self.current_url())
+
+        def dismiss_dialog(sender):
+            self.close_dialogs()
+
+        def confirmed(sender):
+            self.retrieve_url(e_url.get_edit_text())
+            self.close_dialogs()
+
+        dialog = UrlDialogLineBox(
+            urwid.Pile([
+                e_url,
+                urwid.Columns([("weight", 0.45, urwid.Button("Cancel", on_press=dismiss_dialog)), ("weight", 0.1, urwid.Text("")), ("weight", 0.45, urwid.Button("Go", on_press=confirmed))])
+            ]), title="Enter URL"
+        )
+        e_url.confirmed = confirmed
+        dialog.confirmed = confirmed
+        dialog.delegate = self
+        bottom = self.display_widget
+
+        overlay = urwid.Overlay(dialog, bottom, align="center", width=("relative", 65), valign="middle", height="pack", left=2, right=2)
+
+        options = self.delegate.columns.options("weight", self.delegate.right_area_width)
+        self.delegate.columns.contents[1] = (overlay, options)
+        self.delegate.columns.focus_position = 1
+
 
     def load_page(self):
-        load_thread = threading.Thread(target=self.__load)
-        load_thread.setDaemon(True)
-        load_thread.start()
+        cached = self.get_cached(self.current_url())
+        if cached:
+            self.status = Browser.DONE
+            self.page_data = cached
+            self.markup = self.page_data.decode("utf-8")
+            self.attr_maps = markup_to_attrmaps(self.markup, url_delegate=self)
+            
+            self.response_progress = 0
+            self.response_size = None
+            self.response_transfer_size = None
+            self.saved_file_name = None
+            self.loaded_from_cache = True
+
+            self.update_display()
+
+            if not self.history_inc and not self.history_dec and not self.reloading:
+                self.write_history()
+            else:
+                self.history_dec = False
+                self.history_inc = False
+                self.reloading = False
+
+        else:
+            if self.destination_hash != self.loopback:
+                load_thread = threading.Thread(target=self.__load)
+                load_thread.setDaemon(True)
+                load_thread.start()
+            else:
+                RNS.log("Browser handling local page: "+str(self.path), RNS.LOG_DEBUG)
+                page_path = self.app.pagespath+self.path.replace("/page", "", 1)
+
+                page_data = b"The requested local page did not exist in the file system"
+                if os.path.isfile(page_path):
+                    if os.access(page_path, os.X_OK):
+                        generated = subprocess.run([page_path], stdout=subprocess.PIPE)
+                        page_data = generated.stdout
+                    else:
+                        file = open(page_path, "rb")
+                        page_data = file.read()
+                        file.close()
+
+                self.status = Browser.DONE
+                self.page_data = page_data
+                self.markup = self.page_data.decode("utf-8")
+                self.attr_maps = markup_to_attrmaps(self.markup, url_delegate=self)
+                
+                self.response_progress = 0
+                self.response_size = None
+                self.response_transfer_size = None
+                self.saved_file_name = None
+                self.loaded_from_cache = False
+
+                self.update_display()
+
+                if not self.history_inc and not self.history_dec and not self.reloading:
+                    self.write_history()
+                else:
+                    self.history_dec = False
+                    self.history_inc = False
+                    self.reloading = False
 
 
     def __load(self):
         # If an established link exists, but it doesn't match the target
         # destination, we close and clear it.
         if self.link != None and self.link.destination.hash != self.destination_hash:
-            self.link.close()
+            self.link.teardown()
             self.link = None
 
         # If no link to the destination exists, we create one.
@@ -305,14 +559,11 @@ class Browser:
 
             self.link = RNS.Link(destination, established_callback = self.link_established, closed_callback = self.link_closed)
 
-            l_time = time.time()
-            while not self.status == Browser.LINK_ESTABLISHED:
-                now = time.time()
-                if now > l_time+self.timeout:
-                    self.request_timeout()
-                    return
+            while self.status == Browser.ESTABLISHING_LINK:
+                time.sleep(0.1)
 
-                time.sleep(0.25)
+            if self.status != Browser.LINK_ESTABLISHED:
+                return
 
             self.update_display()
 
@@ -321,6 +572,8 @@ class Browser:
         self.response_progress = 0
         self.response_size = None
         self.response_transfer_size = None
+        self.saved_file_name = None
+
 
         self.update_display()
         receipt = self.link.request(
@@ -328,15 +581,17 @@ class Browser:
             data = None,
             response_callback = self.response_received,
             failed_callback = self.request_failed,
-            progress_callback = self.response_progressed,
-            timeout = self.timeout
+            progress_callback = self.response_progressed
         )
 
-        self.last_request_receipt = receipt
-        self.last_request_id = receipt.request_id
+        if receipt:
+            self.last_request_receipt = receipt
+            self.last_request_id = receipt.request_id
+            self.status = Browser.REQUEST_SENT
+            self.update_display()
+        else:
+            self.link.teardown()
 
-        self.status = Browser.REQUEST_SENT
-        self.update_display()
 
 
     def link_established(self, link):
@@ -346,11 +601,22 @@ class Browser:
     def link_closed(self, link):
         if self.status == Browser.DISCONECTED or self.status == Browser.DONE:
             self.link = None
+        elif self.status == Browser.ESTABLISHING_LINK:
+            self.link_establishment_timeout()
         else:
             self.link = None
             self.status = Browser.REQUEST_FAILED
         self.update_display()
 
+    def link_establishment_timeout(self):
+        self.status = Browser.LINK_TIMEOUT
+        self.response_progress = 0
+        self.response_size = None
+        self.response_transfer_size = None
+        self.link = None
+
+        self.update_display()
+
 
     def response_received(self, request_receipt):
         try:
@@ -359,10 +625,121 @@ class Browser:
             self.markup = self.page_data.decode("utf-8")
             self.attr_maps = markup_to_attrmaps(self.markup, url_delegate=self)
             self.response_progress = 0
+            self.loaded_from_cache = False
+
+            # Simple header handling. Should be expanded when more
+            # header tags are added.
+            cache_time = Browser.DEFAULT_CACHE_TIME
+            if self.markup[:4] == "#!c=":
+                endpos = self.markup.find("\n")
+                if endpos == -1:
+                    endpos = len(self.markup)
+                cache_time = int(self.markup[4:endpos])
+
+            self.update_display()
+
+            if not self.history_inc and not self.history_dec and not self.reloading:
+                self.write_history()
+            else:
+                self.history_dec = False
+                self.history_inc = False
+                self.reloading = False
+
+            if cache_time == 0:
+                RNS.log("Received page "+str(self.current_url())+", not caching due to header.", RNS.LOG_DEBUG)
+            else:
+                RNS.log("Received page "+str(self.current_url())+", caching for %.3f hours." % (cache_time/60/60), RNS.LOG_DEBUG)    
+                self.cache_page(cache_time)
+
+        except Exception as e:
+            RNS.log("An error occurred while handling response. The contained exception was: "+str(e))
+
+    def uncache_page(self, url):
+        url_hash = self.url_hash(url)
+        files = os.listdir(self.app.cachepath)
+        for file in files:
+            if file.startswith(url_hash):
+                cachefile = self.app.cachepath+"/"+file
+                os.unlink(cachefile)
+                RNS.log("Removed "+str(cachefile)+" from cache.", RNS.LOG_DEBUG)
+
+    def get_cached(self, url):
+        url_hash = self.url_hash(url)
+        files = os.listdir(self.app.cachepath)
+        for file in files:
+            cachepath = self.app.cachepath+"/"+file
+            try:
+                components = file.split("_")
+                if len(components) == 2 and len(components[0]) == 64 and len(components[1]) > 0:
+                    expires = float(components[1])
+
+                    if time.time() > expires:
+                        RNS.log("Removing stale cache entry "+str(file), RNS.LOG_DEBUG)
+                        os.unlink(cachepath)
+                    else:
+                        if file.startswith(url_hash):
+                            RNS.log("Found "+str(file)+" in cache.", RNS.LOG_DEBUG)
+                            RNS.log("Returning cached page", RNS.LOG_DEBUG)
+                            file = open(cachepath, "rb")
+                            data = file.read()
+                            file.close()
+                            return data
+
+            except Exception as e:
+                RNS.log("Error while parsing cache entry "+str(cachepath)+", removing it.", RNS.LOG_ERROR)
+                RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
+                try:
+                    os.unlink(cachepath)
+                except Exception as e:
+                    RNS.log("Additionally, an exception occurred while unlinking the entry: "+str(e), RNS.LOG_ERROR)
+                    RNS.log("You will probably need to remove this entry manually by deleting the file: "+str(cachepath), RNS.LOG_ERROR)
+
+                
+        return None
+
+
+    def cache_page(self, cache_time):
+        url_hash = self.url_hash(self.current_url())
+        if url_hash == None:
+            RNS.log("Could not cache page "+str(self.current_url()), RNS.LOG_ERROR)
+        else:
+            try:
+                self.uncache_page(self.current_url())
+                cache_expires = time.time()+cache_time
+                filename = url_hash+"_"+str(cache_expires)
+                cachefile = self.app.cachepath+"/"+filename
+                file = open(cachefile, "wb")
+                file.write(self.page_data)
+                file.close()
+                RNS.log("Cached page "+str(self.current_url())+" to "+str(cachefile), RNS.LOG_DEBUG)
+
+            except Exception as e:
+                RNS.log("Could not write cache file for page "+str(self.current_url()), RNS.LOG_ERROR)
+                RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
+
+
+    def file_received(self, request_receipt):
+        try:
+            file_name = request_receipt.response[0]
+            file_data = request_receipt.response[1]
+            file_destination = self.app.downloads_path+"/"+file_name
+            
+            counter = 0
+            while os.path.isfile(file_destination):
+                counter += 1
+                file_destination = self.app.downloads_path+"/"+file_name+"."+str(counter)
+
+            fh = open(file_destination, "wb")
+            fh.write(file_data)
+            fh.close()
+
+            self.saved_file_name = file_destination.replace(self.app.downloads_path+"/", "", 1)
+            self.status = Browser.DONE
+            self.response_progress = 0
 
             self.update_display()
         except Exception as e:
-            RNS.log("An error occurred while handling response. The contained exception was: "+str(e))
+            RNS.log("An error occurred while handling file response. The contained exception was: "+str(e), RNS.LOG_ERROR)
 
     
     def request_failed(self, request_receipt=None):
@@ -394,18 +771,24 @@ class Browser:
 
     def response_progressed(self, request_receipt):
         self.response_progress      = request_receipt.progress
-        self.response_time          = request_receipt.response_time()
+        self.response_time          = request_receipt.get_response_time()
         self.response_size          = request_receipt.response_size
         self.response_transfer_size = request_receipt.response_transfer_size
         self.update_display()
 
 
     def status_text(self):
-        if self.response_transfer_size != None:
-            response_time_str = "{:.2f}".format(self.response_time)
+        if self.status == Browser.DONE and self.response_transfer_size != None:
+            if self.response_time != None:
+                response_time_str = "{:.2f}".format(self.response_time)
+            else:
+                response_time_str = "None"
+
             stats_string = "  "+self.g["page"]+size_str(self.response_size)
             stats_string += "   "+self.g["arrow_d"]+size_str(self.response_transfer_size)+" in "+response_time_str
             stats_string += "s   "+self.g["speed"]+size_str(self.response_transfer_size/self.response_time, suffix="b")+"/s"
+        elif self.loaded_from_cache:
+            stats_string = " (cached)"
         else:
             stats_string = ""
 
@@ -415,6 +798,8 @@ class Browser:
             return "Path requested, waiting for path..."
         elif self.status == Browser.ESTABLISHING_LINK:
             return "Establishing link..."
+        elif self.status == Browser.LINK_TIMEOUT:
+            return "Link establishment timed out"
         elif self.status == Browser.LINK_ESTABLISHED:
             return "Link established"
         elif self.status == Browser.REQUESTING:
@@ -428,7 +813,10 @@ class Browser:
         elif self.status == Browser.RECEIVING_RESPONSE:
             return "Receiving response..."
         elif self.status == Browser.DONE:
-            return "Done"+stats_string
+            if self.saved_file_name == None:
+                return "Done"+stats_string
+            else:
+                return "Saved "+str(self.saved_file_name)+stats_string
         elif self.status == Browser.DISCONECTED:
             return "Disconnected"
         else:
@@ -458,4 +846,18 @@ def size_str(num, suffix='B'):
                 return "%.2f%s%s" % (num, unit, suffix)
         num /= 1000.0
 
-    return "%.2f%s%s" % (num, last_unit, suffix)
+    return "%.2f%s%s" % (num, last_unit, suffix)
+
+class UrlDialogLineBox(urwid.LineBox):
+    def keypress(self, size, key):
+        if key == "esc":
+            self.delegate.close_dialogs()
+        else:
+            return super(UrlDialogLineBox, self).keypress(size, key)
+
+class UrlEdit(urwid.Edit):
+    def keypress(self, size, key):
+        if key == "enter":
+            self.confirmed(self)
+        else:
+            return super(UrlEdit, self).keypress(size, key)    

nomadnet/ui/textui/Config.py

@@ -1,5 +1,6 @@
 import nomadnet
 import urwid
+import platform
 
 class ConfigDisplayShortcuts():
     def __init__(self, app):
@@ -50,6 +51,12 @@ class EditorTerminal(urwid.WidgetWrap):
         self.app = app
         self.parent = parent
         editor_cmd = self.app.config["textui"]["editor"]
+
+        # The "editor" alias is unavailable on Darwin,
+        # so we replace it with nano.
+        if platform.system() == "Darwin" and editor_cmd == "editor":
+            editor_cmd = "nano"
+
         self.term = urwid.Terminal(
             (editor_cmd, self.app.configpath),
             encoding='utf-8',

nomadnet/ui/textui/Conversations.py

@@ -1,4 +1,5 @@
 import RNS
+import os
 import time
 import nomadnet
 import LXMF
@@ -19,7 +20,7 @@ class ConversationDisplayShortcuts():
     def __init__(self, app):
         self.app = app
 
-        self.widget = urwid.AttrMap(urwid.Text("[C-d] Send  [C-k] Clear  [C-w] Close  [C-t] Editor Type  [C-p] Purge"), "shortcutbar")
+        self.widget = urwid.AttrMap(urwid.Text("[C-d] Send  [C-k] Clear Editor  [C-w] Close  [C-t] Editor Type  [C-p] Purge  [C-x] Clear History"), "shortcutbar")
 
 class ConversationsArea(urwid.LineBox):
     def keypress(self, size, key):
@@ -86,7 +87,6 @@ class ConversationsDisplay():
         for conversation in self.app.conversations():
             conversation_list_widgets.append(self.conversation_list_widget(conversation))
 
-        walker = urwid.SimpleFocusListWalker(conversation_list_widgets)
         self.list_widgets = conversation_list_widgets
         self.ilb = IndicativeListBox(
             self.list_widgets,
@@ -316,17 +316,43 @@ class ConversationsDisplay():
             self.ilb.select_item(ilb_position)
         nomadnet.NomadNetworkApp.get_shared_instance().ui.loop.draw_screen()
 
+        if self.currently_displayed_conversation != None:
+            if self.app.conversation_is_unread(self.currently_displayed_conversation):
+                self.app.mark_conversation_read(self.currently_displayed_conversation)
+                try:
+                    if os.path.isfile(self.app.conversationpath + "/" + self.currently_displayed_conversation + "/unread"):
+                        os.unlink(self.app.conversationpath + "/" + self.currently_displayed_conversation + "/unread")
+                except Exception as e:
+                    raise e
+
 
 
 
     def display_conversation(self, sender=None, source_hash=None):
+        if self.currently_displayed_conversation != None:
+            if self.app.conversation_is_unread(self.currently_displayed_conversation):
+                self.app.mark_conversation_read(self.currently_displayed_conversation)
+
         self.currently_displayed_conversation = source_hash
         options = self.widget.options("weight", 1-ConversationsDisplay.list_width)
         self.widget.contents[1] = (self.make_conversation_widget(source_hash), options)
         if source_hash == None:
             self.widget.set_focus_column(0)
         else:
+            if self.app.conversation_is_unread(source_hash):
+                self.app.mark_conversation_read(source_hash)
+                self.update_conversation_list()
+
             self.widget.set_focus_column(1)
+            conversation_position = None
+            index = 0
+            for widget in self.list_widgets:
+                if widget.source_hash == source_hash:
+                    conversation_position = index
+                index += 1
+
+            if conversation_position != None:
+                self.ilb.select_item(conversation_position)
         
 
     def make_conversation_widget(self, source_hash):
@@ -364,6 +390,7 @@ class ConversationsDisplay():
         trust_level  = conversation[2]
         display_name = conversation[1]
         source_hash  = conversation[0]
+        unread       = conversation[4]
 
         g = self.app.ui.glyphs
 
@@ -389,11 +416,17 @@ class ConversationsDisplay():
             focus_style   = "list_focus_untrusted"
 
         display_text = symbol
+
         if display_name != None and display_name != "":
             display_text += " "+display_name
 
         if trust_level != DirectoryEntry.TRUSTED:
             display_text += " <"+source_hash+">"
+        else:
+            if unread:
+                if source_hash != self.currently_displayed_conversation:
+                    display_text += " "+g["unread"]
+
 
         widget = ListEntry(display_text)
         urwid.connect_signal(widget, "click", self.display_conversation, conversation[0])
@@ -544,6 +577,31 @@ class ConversationWidget(urwid.WidgetWrap):
                 
                 urwid.WidgetWrap.__init__(self, self.display_widget)
 
+    def clear_history_dialog(self):
+        def dismiss_dialog(sender):
+            self.dialog_open = False
+            self.conversation_changed(None)
+
+        def confirmed(sender):
+            self.dialog_open = False
+            self.conversation.clear_history()
+            self.conversation_changed(None)
+
+
+        dialog = DialogLineBox(
+            urwid.Pile([
+                urwid.Text("Clear conversation history\n", align="center"),
+                urwid.Columns([("weight", 0.45, urwid.Button("Yes", on_press=confirmed)), ("weight", 0.1, urwid.Text("")), ("weight", 0.45, urwid.Button("No", on_press=dismiss_dialog))])
+            ]), title="?"
+        )
+        dialog.delegate = self
+        bottom = self.messagelist
+
+        overlay = urwid.Overlay(dialog, bottom, align="center", width=34, valign="middle", height="pack", left=2, right=2)
+
+        self.frame.contents["body"] = (overlay, self.frame.options())
+        self.frame.set_focus("body")
+    
     def toggle_editor(self):
         if self.full_editor_active:
             self.frame.contents["footer"] = (self.minimal_editor, None)
@@ -559,7 +617,7 @@ class ConversationWidget(urwid.WidgetWrap):
             if allowed:
                 self.frame.contents["footer"] = (self.minimal_editor, None)
             else:
-                warning = urwid.AttrMap(urwid.Padding(urwid.Text(g["info"]+" You cannot currently communicate with this peer, since it's identity keys are unknown", align="center")), "msg_header_caution")
+                warning = urwid.AttrMap(urwid.Padding(urwid.Text(g["info"]+" You cannot currently communicate with this peer, since it's identity keys are not known", align="center")), "msg_header_caution")
                 self.frame.contents["footer"] = (warning, None)
 
     def toggle_focus_area(self):
@@ -584,6 +642,8 @@ class ConversationWidget(urwid.WidgetWrap):
             self.conversation_changed(None)
         elif key == "ctrl t":
             self.toggle_editor()
+        elif key == "ctrl x":
+            self.clear_history_dialog()
         else:
             return super(ConversationWidget, self).keypress(size, key)
 
@@ -642,9 +702,9 @@ class LXMessageWidget(urwid.WidgetWrap):
         message_time = datetime.fromtimestamp(self.timestamp)
         encryption_string = ""
         if message.get_transport_encrypted():
-            encryption_string = " ["+g["lock"]+" "+str(message.get_transport_encryption())+"]"
+            encryption_string = " ["+g["encrypted"]+" "+str(message.get_transport_encryption())+"]"
         else:
-            encryption_string = " ["+g["unlock"]+" "+str(message.get_transport_encryption())+"]"
+            encryption_string = " ["+g["plaintext"]+" "+str(message.get_transport_encryption())+"]"
         
         title_string = message_time.strftime(time_format)+encryption_string
 

nomadnet/ui/textui/Guide.py

@@ -61,14 +61,16 @@ class SelectText(urwid.Text):
         return True
 
 class GuideEntry(urwid.WidgetWrap):
-    def __init__(self, app, reader, topic_name):
+    def __init__(self, app, parent, reader, topic_name):
         self.app = app
+        self.parent = parent
         self.reader = reader
         self.last_keypress = None
+        self.topic_name = topic_name
         g = self.app.ui.glyphs
 
         widget = ListEntry(topic_name)
-        urwid.connect_signal(widget, "click", self.display_topic, topic_name)
+        urwid.connect_signal(widget, "click", self.display_topic, self.topic_name)
 
         style = "topic_list_normal"
         focus_style = "list_focus"
@@ -79,7 +81,20 @@ class GuideEntry(urwid.WidgetWrap):
         markup = TOPICS[topic]
         attrmaps = markup_to_attrmaps(markup, url_delegate=None)
 
+        topic_position = None
+        index = 0
+        for topic in self.parent.topic_list:
+            widget = topic._original_widget
+            if widget.topic_name == self.topic_name:
+                topic_position = index
+            index += 1
+
+        if topic_position != None:
+            self.parent.ilb.select_item(topic_position)
+
         self.reader.set_content_widgets(attrmaps)
+        self.reader.focus_reader()
+
 
     def micron_released_focus(self):
         self.reader.focus_topics()
@@ -89,17 +104,21 @@ class TopicList(urwid.WidgetWrap):
         self.app = app
         g = self.app.ui.glyphs
 
+        self.first_run_entry = GuideEntry(self.app, self, guide_display, "First Run")
+
         self.topic_list = [
-            GuideEntry(self.app, guide_display, "Introduction"),
-            # GuideEntry(self.app, guide_display, "Conversations"),
-            GuideEntry(self.app, guide_display, "Markup"),
-            GuideEntry(self.app, guide_display, "First Run"),
-            GuideEntry(self.app, guide_display, "Credits & Licenses"),
+            GuideEntry(self.app, self, guide_display, "Introduction"),
+            GuideEntry(self.app, self, guide_display, "Hosting a Node"),
+            GuideEntry(self.app, self, guide_display, "Markup"),
+            self.first_run_entry,
+            GuideEntry(self.app, self, guide_display, "Display Test"),
+            GuideEntry(self.app, self, guide_display, "Credits & Licenses"),
         ]
 
         self.ilb = IndicativeListBox(
             self.topic_list,
             initialization_is_selection_change=False,
+            highlight_offFocus="list_off_focus"
         )
 
         urwid.WidgetWrap.__init__(self, urwid.LineBox(self.ilb, title="Topics"))
@@ -118,7 +137,7 @@ class GuideDisplay():
         self.app = app
         g = self.app.ui.glyphs
 
-        topic_text = urwid.Text("\nNo topic selected", align="left")
+        topic_text = urwid.Text("\n  No topic selected", align="left")
 
         self.left_area  = TopicList(self.app, self)
         self.right_area = urwid.LineBox(urwid.Filler(topic_text, "top"))
@@ -135,6 +154,10 @@ class GuideDisplay():
         self.shortcuts_display = GuideDisplayShortcuts(self.app)
         self.widget = self.columns
 
+        if self.app.firstrun:
+            entry = self.left_area.first_run_entry
+            entry.display_topic(entry.display_topic, entry.topic_name)
+
     def set_content_widgets(self, new_content):
         options = self.columns.options(width_type="weight", width_amount=1-GuideDisplay.list_width)
         pile = urwid.Pile(new_content)
@@ -149,6 +172,9 @@ class GuideDisplay():
     def focus_topics(self):
         self.columns.focus_position = 0
 
+    def focus_reader(self):
+        self.columns.focus_position = 1
+
 
 TOPIC_INTRODUCTION = '''>Nomad Network
 
@@ -157,7 +183,7 @@ TOPIC_INTRODUCTION = '''>Nomad Network
 
 The intention with this program is to provide a tool to that allows you to build private and resilient communications platforms that are in complete control and ownership of the people that use them.
 
-Nomad Network is build on LXMF and Reticulum, which together provides the cryptographic mesh functionality and peer-to-peer message routing that Nomad Network relies on. This foundation also makes it possible to use the program over a very wide variety of communication mediums, from packet radio to gigabit fiber.
+Nomad Network is build on LXMF and Reticulum, which together provides the cryptographic mesh functionality and peer-to-peer message routing that Nomad Network relies on. This foundation also makes it possible to use the program over a very wide variety of communication mediums, from packet radio to fiber.
 
 Nomad Network does not need any connections to the public internet to work. In fact, it doesn't even need an IP or Ethernet network. You can use it entirely over packet radio, LoRa or even serial lines. But if you wish, you can bridge islanded Reticulum networks over the Internet or private ethernet networks, or you can build networks running completely over the Internet. The choice is yours.
 
@@ -195,11 +221,148 @@ If no nodes exist on a network, all peers will still be able to communicate dire
 
 '''
 
+TOPIC_HOSTING = '''>Hosting a Node
+
+To host a node on the network, you must enable it in the configuration file, by setting `*enable_node`* directive to `*yes`*. You should also configure the other node-related parameters such as the node name and announce interval settings. Once node hosting has been enabled in the configuration, Nomad Network will start hosting your node as soon as the program is lauched, and other peers on the network will be able to connect and interact with content on your node.
+
+By default, no content is defined, apart from a short placeholder home page. To learn how to add your own content, read on.
+
+>>Pages
+
+Nomad Network nodes can host pages similar to web pages, that other peers can read and interact with. Pages are written in a compact markup language called `*micron`*. To learn how to write formatted pages with micron, see the `*Markup`* section of this guide (which is, itself, written in micron). Pages can be linked arbitrarily with hyperlinks, that can also link to pages (or other resources) on other nodes.
+
+To add pages to your node, place micron files in the `*pages`* directory of your Nomad Network programs `*storage`* directory. By default, the path to this will be `!~/.nomadnetwork/storage/pages`!. You should probably create the file `!index.mu`! first, as this is the page that will get served by default to a connecting peer.
+
+You can control how long a peer will cache your pages by including the cache header in a page. To do so, the first line of your page must start with `!#!c=X`!, where `!X`! is the cache time in seconds. To tell the peer to always load the page from your node, and never cache it, set the cache time to zero. You should only do this if there is a real need, for example if your page displays dynamic content that `*must`* be updated at every page view. The default caching time is 12 hours. In most cases, you should not need to include the cache control header in your pages.
+
+You can use a preprocessor such as PHP, bash, Python (or whatever you prefer) to generate dynamic pages. To do so, just set executable permissions on the relevant page file, and be sure to include the interpreter at the beginning of the file, for example `!#!/usr/bin/python3`!.
+
+>>Files
+
+Like pages, you can place files you want to make available in the `!~/.nomadnetwork/storage/files`! directory. To let a peer download a file, you should create a link to it in one of your pages.
+
+>>Links and URLs
+
+Links to pages and resources in Nomad Network use a simple URL format. Here is an example:
+
+`!1385edace36466a6b3dd:/page/index.mu`!
+
+The first part is the 10 byte destination address of the node (represented as readable hexadecimal), followed by the `!:`! character. Everything after the `!:`! represents the request path.
+
+By convention, Nomad Network nodes maps all hosted pages under the `!/page`! path, and all hosted files under the `!/file`! path. You can create as many subdirectories for pages and files as you please, and they will be automatically mapped to corresponding request paths.
+
+You can omit the destination address of the node, if you are reffering to a local page or file. You must still keep the `!:`! character. In such a case, the URL to a page could look like this:
+
+`!:/page/other_page.mu`!
+
+The URL to a local file could look like this:
+
+`!:/file/document.pdf`!
+
+Links can be inserted into micron documents. See the `*Markup`* section of this guide for info on how to do so.
+
+'''
+
 TOPIC_CONVERSATIONS = '''>Conversations
 
 Conversations in Nomad Network
 '''
 
+
+TOPIC_FIRST_RUN = '''>First Time Information
+
+Hi there. This first run message will only appear once. It contains a few pointers on getting started with Nomad Network, and getting the most out of the program.
+
+You're currently located in the guide section of the program. I'm sorry I had to drag you here by force, but it will only happen this one time, I promise. If you ever get lost, return here and peruse the list of topics you see on the left. I will do my best to fill it with answers to mostly anything about Nomad Network.
+
+To get the most out of Nomad Network, you will need a terminal that supports UTF-8 and at least 256 colors, ideally true-color. If your terminal supports true-color, you can go to the `![ Config ]`! menu item, launch the editor and change the configuration.
+
+If you don't already have a Nerd Font installed (see https://www.nerdfonts.com/), I also highly recommend to do so, since it will greatly expand the amount of glyphs, icons and graphics that Nomad Network can use. Once you have your terminal set up with a Nerd Font, go to the `![ Config ]`! menu item and enable Nerd Fonts in the configuration instead of normal unicode glyphs.
+
+Nomad Network expects that you are already connected to some form of Reticulum network. That could be as simple as the default UDP-based demo interface on your local ethernet network. This short guide won't go into any details on building, but you will find other entries in the guide that deal with network setup and configuration.
+
+At least, if Nomad Network launches, it means that it is connected to a running Reticulum instance, that should in turn be connected to `*something`*, which should get you started.
+
+For more some more information, you can also read the `*Introduction`* section of this guide.
+
+Now go out there and explore. This is still early days. See what you can find and create.
+
+>>>>>>>>>>>>>>>
+-\u223f
+<
+
+'''
+
+TOPIC_DISPLAYTEST = '''>Markup & Color Display Test
+
+
+`cYou can use this section to gauge how well your terminal reproduces the various types of formatting used by Nomad Network.
+``
+
+
+>>>>>>>>>>>>>>>>>>>>
+-\u223f
+<
+
+
+>>
+`a`!This line should be bold, and aligned to the left`!
+
+`c`*This one should be italic and centered`*
+
+`r`_And this one should be underlined, aligned right`_
+``
+
+The following line should contain a red gradient bar:
+`B100 `B200 `B300 `B400 `B500 `B600 `B700 `B800 `B900 `Ba00 `Bb00 `Bc00 `Bd00 `Be00 `Bf00`b
+
+The following line should contain a green gradient bar:
+`B010 `B020 `B030 `B040 `B050 `B060 `B070 `B080 `B090 `B0a0 `B0b0 `B0c0 `B0d0 `B0e0 `B0f0`b
+
+The following line should contain a blue gradient bar:
+`B001 `B002 `B003 `B004 `B005 `B006 `B007 `B008 `B009 `B00a `B00b `B00c `B00d `B00e `B00f`b
+
+Unicode Glyphs   : \u2713  \u2715  \u26a0  \u24c3  \u2193
+
+Nerd Font Glyphs : \uf484  \uf9c4 \uf719  \uf502  \uf415  \uf023  \uf06e
+'''
+
+
+TOPIC_LICENSES = '''>Thanks, Acknowledgements and Licenses
+
+This program uses various other software components, without which Nomad Network would not have been possible. Sincere thanks to the authors and contributors of the following projects
+
+>>>
+ - `!Cryptography.io`! by `*pyca`*
+   https://cryptography.io/
+   BSD License
+
+ - `!Urwid`! by `*Ian Ward`*
+   http://urwid.org/
+   LGPL-2.1 License
+
+ - `!Additional Urwid Widgets`! by `*AFoeee`*
+   https://github.com/AFoeee/additional_urwid_widgets
+   MIT License
+
+ - `!Scrollable`! by `*rndusr`*
+   https://github.com/rndusr/stig/blob/master/stig/tui/scroll.py
+   GPLv3 License
+
+ - `!Configobj`! by `*Michael Foord`*
+   https://github.com/DiffSK/configobj
+   BSD License
+
+ - `!Reticulum Network Stack`! by `*unsignedmark`*
+   https://github.com/markqvist/Reticulum
+   MIT License
+
+ - `!LXMF`! by `*unsignedmark`*
+   https://github.com/markqvist/LXMF
+   MIT License
+'''
+
+
 TOPIC_MARKUP = '''>Outputting Formatted Text
 
 
@@ -398,6 +561,59 @@ You can use `B5d5`F222 color `f`B333 `Ff00f`Ff80o`Ffd0r`F9f0m`F0f2a`F0fdt`F07ft`
 ``
 
 
+>Links
+
+Links to pages, files or other resources can be created with the \\`[ tag, which should always be terminated with a closing ]. You can create links with and without labels, it is up to you to control the formatting of links with other tags. Although not strictly necessary, it is good practice to at least format links with underlining.
+
+Here's a few examples:
+
+`Faaa
+`=
+Here is a link without any label: `[1385edace36466a6b3dd:/page/index.mu]
+
+This is a `[labeled link`1385edace36466a6b3dd:/page/index.mu] to the same page, but it's hard to see if you don't know it
+
+Here is `F00a`_`[a more visible link`1385edace36466a6b3dd:/page/index.mu]`_`f
+`=
+``
+
+The above markup produces the following output:
+
+`Faaa`B333
+
+Here is a link without any label: `[1385edace36466a6b3dd:/page/index.mu]
+
+This is a `[labeled link`1385edace36466a6b3dd:/page/index.mu] to the same page, but it's hard to see if you don't know it
+
+Here is `F00f`_`[a more visible link`1385edace36466a6b3dd:/page/index.mu]`_`f
+
+``
+
+When links like these are displayed in the built-in browser, clicking on them or activating them using the keyboard will cause the browser to load the specified URL.
+
+>Comments
+
+You can insert comments that will not be displayed in the output by starting a line with the # character.
+
+Here's an example:
+
+`Faaa
+`=
+# This line will not be displayed
+This line will
+`=
+``
+
+The above markup produces the following output:
+
+`Faaa`B333
+
+# This line will not be displayed
+This line will
+
+``
+
+
 >Literals
 
 To display literal content, for example source-code, or blocks of text that should not be interpreted by micron, you can use literal blocks, specified by the \\`= tag. Below is the source code of this entire document, presented as a literal block.
@@ -409,70 +625,13 @@ To display literal content, for example source-code, or blocks of text that shou
 TOPIC_MARKUP += TOPIC_MARKUP.replace("`=", "\\`=") + "[ micron source for document goes here, we don't want infinite recursion now, do we? ]\n\\`="
 TOPIC_MARKUP += "\n`=\n\n>Closing Remarks\n\nIf you made it all the way here, you should be well equipped to write documents and pages using micron. Thank you for staying with me.\n\n`c\U0001F332\n"
 
-TOPIC_FIRST_RUN = '''>First Time Information
-
-Hi there. This first run message will only appear once. It contains a few pointers on getting started with Nomad Network, and getting the most out of the program.
-
-You're currently located in the guide section of the program. I'm sorry I had to drag you here by force, but it will only happen this one time, I promise. If you ever get lost, return here and peruse the list of topics you see on the left. I will do my best to fill it with answers to mostly anything about Nomad Network.
-
-To get the most out of Nomad Network, you will need a terminal that supports UTF-8 and at least 256 colors, ideally true-color.
-
-By default, Nomad Network starts in low-color mode. It does this for the sake of compatibility, but it does look rather ugly. If your terminal supports true-color or just 256 colors, you should go to the `![ Config ]`! menu item, launch the editor and change the configuration to use a high-color mode.
-
-If you don't already have a Nerd Font installed (see https://www.nerdfonts.com/), I also highly recommend to do so, since it will greatly expand the amount of glyphs, icons and graphics that Nomad Network can use.
-
-Nomad Network expects that you are already connected to some form of Reticulum network. That could be as simple as the default UDP-based demo interface on your local ethernet network. This short guide won't go into any details on building, but you will find other entries in the guide that deal with network setup and configuration.
-
-At least, if Nomad Network launches, it means that it is connected to a running Reticulum instance, that should in turn be connected to `*something`*, which should get you started.
-
-For more some more information, you can also read the `*Introduction`* section of this guide.
-
-Now go out there and explore. This is still early days. See what you can find and create.
-
->>>>>>>>>>>>>>>
--\u223f
-<
-
-'''
-
-TOPIC_LICENSES = '''>Thanks, Acknowledgements and Licenses
-
-This program uses various other software components, without which Nomad Network would not have been possible. Sincere thanks to the authors and contributors of the following projects
-
->>>
- - `!Cryptography.io`! by `*pyca`*
-   https://cryptography.io/
-   BSD License
-
- - `!Urwid`! by `*Ian Ward`*
-   http://urwid.org/
-   LGPL-2.1 License
-
- - `!Additional Urwid Widgets`! by `*AFoeee`*
-   https://github.com/AFoeee/additional_urwid_widgets
-   MIT License
-
- - `!Scrollable`! by `*rndusr`*
-   https://github.com/rndusr/stig/blob/master/stig/tui/scroll.py
-   GPLv3 License
-
- - `!Configobj`! by `*Michael Foord`*
-   https://github.com/DiffSK/configobj
-   BSD License
-
- - `!Reticulum Network Stack`! by `*unsignedmark`*
-   https://github.com/markqvist/Reticulum
-   MIT License
-
- - `!LXMF`! by `*unsignedmark`*
-   https://github.com/markqvist/LXMF
-   MIT License
-'''
 
 TOPICS = {
     "Introduction": TOPIC_INTRODUCTION,
     "Conversations": TOPIC_CONVERSATIONS,
+    "Hosting a Node": TOPIC_HOSTING,
     "Markup": TOPIC_MARKUP,
     "First Run": TOPIC_FIRST_RUN,
+    "Display Test": TOPIC_DISPLAYTEST,
     "Credits & Licenses": TOPIC_LICENSES,
 }

nomadnet/ui/textui/Main.py

@@ -21,7 +21,10 @@ class SubDisplays():
         self.log_display = LogDisplay(self.app)
         self.guide_display = GuideDisplay(self.app)
 
-        self.active_display = self.conversations_display
+        if app.firstrun:
+            self.active_display = self.guide_display
+        else:
+            self.active_display = self.conversations_display
 
     def active(self):
         return self.active_display

nomadnet/ui/textui/MicronParser.py

@@ -5,16 +5,26 @@ from urwid.util import is_mouse_press
 from urwid.text_layout import calc_coords
 import re
 
-DEFAULT_FG = "ddd"
+DEFAULT_FG_DARK  = "ddd"
+DEFAULT_FG_LIGHT = "222"
 DEFAULT_BG = "default"
 
-STYLES = {
-    "plain":    { "fg": DEFAULT_FG, "bg": DEFAULT_BG, "bold": False, "underline": False, "italic": False },
+SELECTED_STYLES = None
+
+STYLES_DARK = {
+    "plain":    { "fg": DEFAULT_FG_DARK, "bg": DEFAULT_BG, "bold": False, "underline": False, "italic": False },
     "heading1": { "fg": "222", "bg": "bbb", "bold": False, "underline": False, "italic": False },
     "heading2": { "fg": "111", "bg": "999", "bold": False, "underline": False, "italic": False },
     "heading3": { "fg": "000", "bg": "777", "bold": False, "underline": False, "italic": False },
 }
 
+STYLES_LIGHT = {
+    "plain":    { "fg": DEFAULT_FG_LIGHT, "bg": DEFAULT_BG, "bold": False, "underline": False, "italic": False },
+    "heading1": { "fg": "000", "bg": "777", "bold": False, "underline": False, "italic": False },
+    "heading2": { "fg": "111", "bg": "aaa", "bold": False, "underline": False, "italic": False },
+    "heading3": { "fg": "222", "bg": "ccc", "bold": False, "underline": False, "italic": False },
+}
+
 SYNTH_STYLES = []
 SYNTH_SPECS  = {}
 
@@ -22,12 +32,18 @@ SECTION_INDENT = 2
 INDENT_RIGHT   = 1
 
 def markup_to_attrmaps(markup, url_delegate = None):
+    global SELECTED_STYLES
+    if nomadnet.NomadNetworkApp.get_shared_instance().config["textui"]["theme"] == nomadnet.ui.TextUI.THEME_DARK:
+        SELECTED_STYLES = STYLES_DARK
+    else:
+        SELECTED_STYLES = STYLES_LIGHT
+
     attrmaps = []
 
     state = {
         "literal": False,
         "depth": 0,
-        "fg_color": DEFAULT_FG,
+        "fg_color": SELECTED_STYLES["plain"]["fg"],
         "bg_color": DEFAULT_BG,
         "formatting": {
             "bold": False,
@@ -72,6 +88,10 @@ def parse_line(line, state, url_delegate):
             if first_char == "\\":
                 line = line[1:]
 
+            # Check for comments
+            elif first_char == "#":
+                return None
+
             # Check for section heading reset
             elif first_char == "<":
                 state["depth"] = 0
@@ -86,8 +106,8 @@ def parse_line(line, state, url_delegate):
                 
                     for j in range(1, i+1):
                         wanted_style = "heading"+str(i)
-                        if wanted_style in STYLES:
-                            style = STYLES[wanted_style]
+                        if wanted_style in SELECTED_STYLES:
+                            style = SELECTED_STYLES[wanted_style]
 
                 line = line[state["depth"]:]
                 if len(line) > 0:
@@ -229,7 +249,7 @@ def make_output(state, line, url_delegate):
                             state["fg_color"] = color
                             skip = 3
                     elif c == "f":
-                        state["fg_color"] = DEFAULT_FG
+                        state["fg_color"] = SELECTED_STYLES["plain"]["fg"]
                     elif c == "B":
                         if len(line) >= i+4:
                             color = line[i+1:i+4]
@@ -241,7 +261,7 @@ def make_output(state, line, url_delegate):
                         state["formatting"]["bold"]      = False 
                         state["formatting"]["underline"] = False
                         state["formatting"]["italic"]    = False
-                        state["fg_color"] = DEFAULT_FG
+                        state["fg_color"] = SELECTED_STYLES["plain"]["fg"]
                         state["bg_color"] = DEFAULT_BG
                         state["align"] = state["default_align"]
                     elif c == "c":
@@ -393,6 +413,17 @@ class LinkableText(urwid.Text):
 
         return None
 
+    def peek_link(self):
+        item = self.find_item_at_pos(self._cursor_position)
+        if item != None:
+            if isinstance(item, LinkSpec):
+                if self.delegate != None:
+                    self.delegate.marked_link(item.link_target)
+            else:
+                if self.delegate != None:
+                    self.delegate.marked_link(None)
+
+
     def keypress(self, size, key):
         part_positions = [0]
         parts = []
@@ -449,6 +480,9 @@ class LinkableText(urwid.Text):
         if focus and (self.delegate == None or now < self.delegate.last_keypress+self.key_timeout):
             c = urwid.CompositeCanvas(c)
             c.cursor = self.get_cursor_coords(size)
+            if self.delegate != None:
+                self.peek_link()
+
         return c
 
     def get_cursor_coords(self, size):

nomadnet/ui/textui/Network.py

@@ -13,7 +13,7 @@ class NetworkDisplayShortcuts():
         self.app = app
         g = app.ui.glyphs
 
-        self.widget = urwid.AttrMap(urwid.Text("[C-l] Toggle Nodes/Announces view  [C-x] Remove entry  [C-w] Disconnect remote"), "shortcutbar")
+        self.widget = urwid.AttrMap(urwid.Text("[C-l] Toggle Nodes/Announces view  [C-x] Remove entry  [C-w] Disconnect [C-d] Back  [C-f] Forward  [C-r] Reload  [C-u] Enter URL"), "shortcutbar")
         #   "[C-"+g["arrow_u"]+g["arrow_d"]+"] Navigate Lists"
 
 
@@ -67,9 +67,9 @@ class AnnounceInfo(urwid.WidgetWrap):
         is_node      = announce[3]
 
         if is_node:
-            type_string = g["node"] + " Node"
+            type_string = "Node " + g["node"]
         else:
-            type_string = g["peer"] + " Peer"
+            type_string = "Peer " + g["peer"]
 
         try:
             data_str = announce[2].decode("utf-8")
@@ -461,8 +461,6 @@ class KnownNodes(urwid.WidgetWrap):
         # TODO: Sort list
         return widget_list
 
-
-
 class NodeEntry(urwid.WidgetWrap):
     def __init__(self, app, node, delegate):
         source_hash = node.source_hash
@@ -537,6 +535,69 @@ class AnnounceTime(urwid.WidgetWrap):
         self.started = False
 
 
+class NodeAnnounceTime(urwid.WidgetWrap):
+    def __init__(self, app):
+        self.started = False
+        self.app = app
+        self.timeout = self.app.config["textui"]["animation_interval"]
+        self.display_widget = urwid.Text("")
+        self.update_time()
+
+        urwid.WidgetWrap.__init__(self, self.display_widget)
+
+    def update_time(self):
+        self.last_announce_string = "Never"
+        if self.app.peer_settings["node_last_announce"] != None:
+            self.last_announce_string = pretty_date(int(self.app.peer_settings["node_last_announce"]))
+
+        self.display_widget.set_text("Last Announce   : "+self.last_announce_string)
+
+    def update_time_callback(self, loop=None, user_data=None):
+        self.update_time()
+        if self.started:
+            self.app.ui.loop.set_alarm_in(self.timeout, self.update_time_callback)
+
+    def start(self):
+        was_started = self.started
+        self.started = True
+        if not was_started:
+            self.update_time_callback()
+
+    def stop(self):
+        self.started = False
+
+class NodeActiveConnections(urwid.WidgetWrap):
+    def __init__(self, app):
+        self.started = False
+        self.app = app
+        self.timeout = self.app.config["textui"]["animation_interval"]
+        self.display_widget = urwid.Text("")
+        self.update_stat()
+
+        urwid.WidgetWrap.__init__(self, self.display_widget)
+
+    def update_stat(self):
+        self.stat_string = "None"
+        if self.app.node != None:
+            self.stat_string = str(len(self.app.node.destination.links))
+
+        self.display_widget.set_text("Connected Peers : "+self.stat_string)
+
+    def update_stat_callback(self, loop=None, user_data=None):
+        self.update_stat()
+        if self.started:
+            self.app.ui.loop.set_alarm_in(self.timeout, self.update_stat_callback)
+
+    def start(self):
+        was_started = self.started
+        self.started = True
+        if not was_started:
+            self.update_stat_callback()
+
+    def stop(self):
+        self.started = False
+
+
 class LocalPeer(urwid.WidgetWrap):
     announce_timer = None
 
@@ -598,9 +659,9 @@ class LocalPeer(urwid.WidgetWrap):
             options = self.parent.left_pile.options(height_type="pack", height_amount=None)
             self.parent.left_pile.contents[2] = (overlay, options)
 
-        def node_settings_query(sender):
+        def node_info_query(sender):
             options = self.parent.left_pile.options(height_type="pack", height_amount=None)
-            self.parent.left_pile.contents[2] = (self.parent.node_settings_display, options)
+            self.parent.left_pile.contents[2] = (self.parent.node_info_display, options)
 
         if LocalPeer.announce_timer == None:
             self.t_last_announce = AnnounceTime(self.app)
@@ -619,7 +680,7 @@ class LocalPeer(urwid.WidgetWrap):
                 self.t_last_announce,
                 announce_button,
                 urwid.Divider(g["divider1"]),
-                urwid.Columns([("weight", 0.45, urwid.Button("Save", on_press=save_query)), ("weight", 0.1, urwid.Text("")), ("weight", 0.45, urwid.Button("Node Cfg", on_press=node_settings_query))])
+                urwid.Columns([("weight", 0.45, urwid.Button("Save", on_press=save_query)), ("weight", 0.1, urwid.Text("")), ("weight", 0.45, urwid.Button("Node Info", on_press=node_info_query))])
             ]
         )
 
@@ -629,26 +690,102 @@ class LocalPeer(urwid.WidgetWrap):
         self.t_last_announce.start()
 
 
-class NodeSettings(urwid.WidgetWrap):
+class NodeInfo(urwid.WidgetWrap):
+    announce_timer = None
+    links_timer = None
+
     def __init__(self, app, parent):
         self.app = app
         self.parent = parent
         g = self.app.ui.glyphs
 
+        self.dialog_open = False
+        display_name = self.app.node.name
+        if display_name == None:
+            display_name = ""
+
+        t_id = urwid.Text("Addr : "+RNS.hexrep(self.app.node.destination.hash, delimit=False))
+        e_name = urwid.Text("Name : "+display_name)
+
+        def announce_query(sender):
+            def dismiss_dialog(sender):
+                self.dialog_open = False
+                options = self.parent.left_pile.options(height_type="pack", height_amount=None)
+                self.parent.left_pile.contents[2] = (NodeInfo(self.app, self.parent), options)
+
+            self.app.node.announce()
+
+            dialog = DialogLineBox(
+                urwid.Pile([
+                    urwid.Text("\n\n\nAnnounce Sent\n\n", align="center"),
+                    urwid.Button("OK", on_press=dismiss_dialog)
+                ]), title=g["info"]
+            )
+            dialog.delegate = self
+            bottom = self
+
+            #overlay = urwid.Overlay(dialog, bottom, align="center", width=("relative", 100), valign="middle", height="pack", left=4, right=4)
+            overlay = dialog
+            
+            self.dialog_open = True
+            options = self.parent.left_pile.options(height_type="pack", height_amount=None)
+            self.parent.left_pile.contents[2] = (overlay, options)
+
         def show_peer_info(sender):
             options = self.parent.left_pile.options(height_type="pack", height_amount=None)
             self.parent.left_pile.contents[2] = (LocalPeer(self.app, self.parent), options)
 
-        widget_style = "inactive_text"
-        pile = urwid.Pile([
-            urwid.Text("\n"+g["info"], align="center"),
-            urwid.Text("\nNode Hosting currently unavailable\n\n", align="center"),
-            urwid.Padding(urwid.Button("Back", on_press=show_peer_info), "center", "pack")
-        ])
+        def connect_query(sender):
+            self.parent.browser.retrieve_url(RNS.hexrep(self.app.node.destination.hash, delimit=False))
+
+        if NodeInfo.announce_timer == None:
+            self.t_last_announce = NodeAnnounceTime(self.app)
+            NodeInfo.announce_timer = self.t_last_announce
+        else:
+            self.t_last_announce = NodeInfo.announce_timer
+            self.t_last_announce.update_time()
+
+        if NodeInfo.links_timer == None:
+            self.t_active_links = NodeActiveConnections(self.app)
+            NodeInfo.links_timer = self.t_active_links
+        else:
+            self.t_active_links = NodeInfo.links_timer
+            self.t_active_links.update_stat()
+
+        announce_button = urwid.Button("Announce Now", on_press=announce_query)
+        connect_button = urwid.Button("Browse", on_press=connect_query)
+
+        widget_style = ""
+        if self.app.enable_node:
+            pile = urwid.Pile([
+                t_id,
+                e_name,
+                urwid.Divider(g["divider1"]),
+                self.t_last_announce,
+                self.t_active_links,
+                urwid.Divider(g["divider1"]),
+                urwid.Columns([
+                    ("weight", 0.3, urwid.Button("Back", on_press=show_peer_info)),
+                    ("weight", 0.1, urwid.Text("")),
+                    ("weight", 0.3, connect_button),
+                    ("weight", 0.1, urwid.Text("")),
+                    ("weight", 0.3, announce_button)
+                ])
+            ])
+        else:
+            pile = urwid.Pile([
+                urwid.Text("\n"+g["info"], align="center"),
+                urwid.Text("\nThis instance is not hosting a node\n\n", align="center"),
+                urwid.Padding(urwid.Button("Back", on_press=show_peer_info), "center", "pack")
+            ])
 
         self.display_widget = pile
 
-        urwid.WidgetWrap.__init__(self, urwid.AttrMap(urwid.LineBox(self.display_widget, title="Node Settings"), widget_style))
+        urwid.WidgetWrap.__init__(self, urwid.AttrMap(urwid.LineBox(self.display_widget, title="Local Node Info"), widget_style))
+
+    def start(self):
+        self.t_last_announce.start()
+        self.t_active_links.start()
 
 
 
@@ -712,14 +849,14 @@ class NetworkStats(urwid.WidgetWrap):
         self.w_heard_peers.start()
         self.w_known_nodes.start()
 
-
-
 class NetworkLeftPile(urwid.Pile):
     def keypress(self, size, key):
         if key == "ctrl l":
             self.parent.toggle_list()
         elif key == "ctrl w":
             self.parent.browser.disconnect()
+        elif key == "ctrl u":
+            self.parent.browser.url_dialog()
         else:
             return super(NetworkLeftPile, self).keypress(size, key)
 
@@ -733,11 +870,14 @@ class NetworkDisplay():
 
         self.browser = Browser(self.app, "nomadnetwork", "node", auth_identity = self.app.identity, delegate = self)
 
+        if self.app.node != None:
+            self.browser.loopback = self.app.node.destination.hash
+
         self.known_nodes_display = KnownNodes(self.app)
         self.network_stats_display = NetworkStats(self.app, self)
         self.announce_stream_display = AnnounceStream(self.app, self)
         self.local_peer_display = LocalPeer(self.app, self)
-        self.node_settings_display = NodeSettings(self.app, self)
+        self.node_info_display = NodeInfo(self.app, self)
 
         self.known_nodes_display.delegate = self
 
@@ -752,11 +892,12 @@ class NetworkDisplay():
 
         self.left_area = self.left_pile
         self.right_area = self.browser.display_widget
+        self.right_area_width = 1-NetworkDisplay.list_width
 
         self.columns = urwid.Columns(
             [
                 ("weight", NetworkDisplay.list_width, self.left_area),
-                ("weight", 1-NetworkDisplay.list_width, self.right_area)
+                ("weight", self.right_area_width, self.right_area)
             ],
             dividechars=0, focus_column=0
         )
@@ -793,8 +934,12 @@ class NetworkDisplay():
 
     def start(self):
         self.local_peer_display.start()
+        self.node_info_display.start()
         self.network_stats_display.start()
-        self.announce_stream_display.start()
+        # There seems to be an intermittent memory leak somewhere
+        # in the periodic updating here. The periodic updater should
+        # not be needed anymore, so dis
+        #self.announce_stream_display.start()
 
     def shortcuts(self):
         return self.shortcuts_display

nomadnet/vendor/Scrollable.py

@@ -31,7 +31,7 @@ class Scrollable(urwid.WidgetDecoration):
     def selectable(self):
         return True
 
-    def __init__(self, widget):
+    def __init__(self, widget, force_forward_keypress = False):
         """Box widget that makes a fixed or flow widget vertically scrollable
 
         TODO: Focusable widgets are handled, including switching focus, but
@@ -49,6 +49,7 @@ class Scrollable(urwid.WidgetDecoration):
         self._forward_keypress = None
         self._old_cursor_coords = None
         self._rows_max_cached = 0
+        self.force_forward_keypress = force_forward_keypress
         self.__super.__init__(widget)
 
     def render(self, size, focus=False):
@@ -127,7 +128,7 @@ class Scrollable(urwid.WidgetDecoration):
 
     def keypress(self, size, key):
         # Maybe offer key to original widget
-        if self._forward_keypress:
+        if self._forward_keypress or self.force_forward_keypress:
             ow = self._original_widget
             ow_size = self._get_original_widget_size(size)
 

nomadnet/vendor/quotes.py

@@ -1,4 +1,5 @@
 quotes = [
     ("I want the wisdom that wise men revere. I want more.", "Faithless"),
-    ("That's enough entropy for you my friend", "Unknown")
+    ("That's enough entropy for you my friend", "Unknown"),
+    ("Any time two people connect online, it's financed by a third person who believes they can manipulate the first two", "Jaron Lanier")
 ]

setup.py

@@ -23,6 +23,6 @@ setuptools.setup(
     entry_points= {
         'console_scripts': ['nomadnet=nomadnet.nomadnet:main']
     },
-    install_requires=['rns>=0.2.3', 'lxmf>=0.0.8', 'urwid>=2.1.2'],
+    install_requires=['rns>=0.2.5', 'lxmf>=0.0.9', 'urwid>=2.1.2'],
     python_requires='>=3.6',
 )