Updated guide. Implemented link peeking.

README.md

@@ -3,28 +3,29 @@ Nomad Network
 
 Communicate Freely.
 
-![Screenshot](https://github.com/markqvist/NomadNet/raw/master/docs/screenshots/3.png)
+![Screenshot](https://github.com/markqvist/NomadNet/raw/master/docs/screenshots/1.png)
 
 The intention with this program is to provide a tool to that allows you to build private and resilient communications platforms that are in complete control and ownership of the people that use them.
 
-Nomad Network is build on [LXMF](https://github.com/markqvist/LXMF) and [Reticulum](https://github.com/markqvist/Reticulum), which together provides the cryptographic mesh functionality and peer-to-peer message routing that Nomad Network relies on. This foundation also makes it possible to use the program over a very wide variety of communication mediums, from packet radio to gigabit fiber.
+Nomad Network is build on [LXMF](https://github.com/markqvist/LXMF) and [Reticulum](https://github.com/markqvist/Reticulum), which together provides the cryptographic mesh functionality and peer-to-peer message routing that Nomad Network relies on. This foundation also makes it possible to use the program over a very wide variety of communication mediums, from packet radio to fiber.
 
 Nomad Network does not need any connections to the public internet to work. In fact, it doesn't even need an IP or Ethernet network. You can use it entirely over packet radio, LoRa or even serial lines. But if you wish, you can bridge islanded Reticulum networks over the Internet or private ethernet networks, or you can build networks running completely over the Internet. The choice is yours.
 
 ## Notable Features
  - Encrypted messaging over packet-radio, LoRa, WiFi or anything else [Reticulum](https://github.com/markqvist/Reticulum) supports.
  - Zero-configuration, minimal-infrastructure mesh communication
+ - Connectable nodes that can host pages and files
+ - Built-in text-based browser for interacting with contents on nodes
+ - An easy to use and bandwidth efficient markup language for writing pages
 
 ## Current Status
+The current version of the program should be considered a beta release. The program works well, but there will most probably be bugs and possibly sub-optimal performance in some scenarios. On the other hand, this is the ideal time to have an influence on the direction of the development of Nomad Network. To do so, join the discussion, report bugs and request features here on the GitHub project.
 
-
+### Feature roadmap
-The current version of the program should be considered an alpha release. The program works well, but there will most probably be bugs and possibly sub-optimal performance in some scenarios. On the other hand, this is the ideal time to have an influence on the direction of the development of Nomad Network. To do so, join the discussion, report bugs and request features here on the GitHub project.
+ - Page caching in browser
-
+ - Node-side generated pages with PHP, Python, bash or others
-Development is ongoing and current features being implemented are:
+ - Access control and authentication for pages and files
-
  - Propagated messaging and discussion threads
- - Connectable nodes that can host pages, files and other resources
- - Collaborative information sharing and spatial map-style "wikis"
 
 ## Dependencies:
  - Python 3

nomadnet/Conversation.py

@@ -7,6 +7,7 @@ from nomadnet.Directory import DirectoryEntry
 
 class Conversation:
     cached_conversations = {}
+    unread_conversations = {}
     created_callback = None
 
     aspect_filter = "lxmf.delivery"
@@ -26,7 +27,7 @@ class Conversation:
 
         # Add the announce to the directory announce
         # stream logger
-        app.directory.announce_received(destination_hash, app_data)
+        app.directory.lxmf_announce_received(destination_hash, app_data)
 
     @staticmethod
     def query_for_peer(source_hash):
@@ -57,6 +58,16 @@ class Conversation:
             conversation = Conversation.cached_conversations[RNS.hexrep(source_hash, delimit=False)]
             conversation.scan_storage()
 
+        if not source_hash in Conversation.unread_conversations:
+            Conversation.unread_conversations[source_hash] = True
+            try:
+                dirname = RNS.hexrep(source_hash, delimit=False)
+                open(app.conversationpath + "/" + dirname + "/unread", 'a').close()
+            except Exception as e:
+                pass
+
+            Conversation.created_callback()
+
         return ingested_path
 
     @staticmethod
@@ -70,6 +81,13 @@ class Conversation:
                     app_data         = RNS.Identity.recall_app_data(source_hash)
                     display_name     = app.directory.display_name(source_hash)
 
+                    unread = False
+                    if source_hash in Conversation.unread_conversations:
+                        unread = True
+                    elif os.path.isfile(app.conversationpath + "/" + dirname + "/unread"):
+                        Conversation.unread_conversations[source_hash] = True
+                        unread = True
+
                     if display_name == None and app_data:
                         display_name = app_data.decode("utf-8")
 
@@ -80,7 +98,7 @@ class Conversation:
                     
                     trust_level      = app.directory.trust_level(source_hash, display_name)
                     
-                    entry = (source_hash_text, display_name, trust_level, sort_name)
+                    entry = (source_hash_text, display_name, trust_level, sort_name, unread)
                     conversations.append(entry)
 
                 except Exception as e:
@@ -114,6 +132,7 @@ class Conversation:
         self.source_known       = False
         self.source_trusted     = False
         self.source_blocked     = False
+        self.unread             = False
 
         self.__changed_callback = None
 
@@ -143,6 +162,11 @@ class Conversation:
                 message_path = self.messages_path + "/" + filename
                 self.messages.append(ConversationMessage(message_path))
 
+        new_len = len(self.messages)
+
+        if new_len > old_len:
+            self.unread = True
+
         if self.__changed_callback != None:
             self.__changed_callback(self)
 
@@ -156,6 +180,15 @@ class Conversation:
         for purged_message in purged_messages:
             self.messages.remove(purged_message)
 
+    def clear_history(self):
+        purged_messages = []
+        for conversation_message in self.messages:
+            purged_messages.append(conversation_message)
+            conversation_message.purge()
+
+        for purged_message in purged_messages:
+            self.messages.remove(purged_message)
+
     def register_changed_callback(self, callback):
         self.__changed_callback = callback
 

nomadnet/Directory.py

@@ -2,10 +2,22 @@ import os
 import RNS
 import LXMF
 import time
+import nomadnet
 import RNS.vendor.umsgpack as msgpack
 
 class Directory:
-    ANNOUNCE_STREAM_MAXLENGTH = 256
+    ANNOUNCE_STREAM_MAXLENGTH = 64
+
+    aspect_filter = "nomadnetwork.node"
+    @staticmethod
+    def received_announce(destination_hash, announced_identity, app_data):
+        app = nomadnet.NomadNetworkApp.get_shared_instance()
+        destination_hash_text = RNS.hexrep(destination_hash, delimit=False)
+
+        associated_peer = RNS.Destination.hash_from_name_and_identity("lxmf.delivery", announced_identity)
+
+        app.directory.node_announce_received(destination_hash, app_data, associated_peer)
+
 
     def __init__(self, app):
         self.directory_entries = {}
@@ -13,6 +25,7 @@ class Directory:
         self.app = app
         self.load_from_disk()
 
+
     def save_to_disk(self):
         try:
             packed_list = []
@@ -49,16 +62,41 @@ class Directory:
                     entries[e[0]] = DirectoryEntry(e[0], e[1], e[2], hosts_node)
 
                 self.directory_entries = entries
+
                 self.announce_stream = unpacked_directory["announce_stream"]
 
+
             except Exception as e:
                 RNS.log("Could not load directory from disk. The contained exception was: "+str(e), RNS.LOG_ERROR)
 
-    def announce_received(self, source_hash, app_data):
+    def lxmf_announce_received(self, source_hash, app_data):
-        timestamp = time.time()
+        if app_data != None:
-        self.announce_stream.insert(0, (timestamp, source_hash, app_data))
+            timestamp = time.time()
-        while len(self.announce_stream) > Directory.ANNOUNCE_STREAM_MAXLENGTH:
+            self.announce_stream.insert(0, (timestamp, source_hash, app_data, False))
-            self.announce_stream.pop()
+            while len(self.announce_stream) > Directory.ANNOUNCE_STREAM_MAXLENGTH:
+                self.announce_stream.pop()
+            self.app.ui.main_display.sub_displays.network_display.directory_change_callback()
+
+    def node_announce_received(self, source_hash, app_data, associated_peer):
+        if app_data != None:
+            timestamp = time.time()
+            self.announce_stream.insert(0, (timestamp, source_hash, app_data, True))
+            while len(self.announce_stream) > Directory.ANNOUNCE_STREAM_MAXLENGTH:
+                self.announce_stream.pop()
+
+            if self.trust_level(associated_peer) == DirectoryEntry.TRUSTED:
+                node_entry = DirectoryEntry(source_hash, display_name=app_data.decode("utf-8"), trust_level=DirectoryEntry.TRUSTED, hosts_node=True)
+                self.remember(node_entry)
+                self.app.ui.main_display.sub_displays.network_display.directory_change_callback()
+
+    def remove_announce_with_timestamp(self, timestamp):
+        selected_announce = None
+        for announce in self.announce_stream:
+            if announce[0] == timestamp:
+                selected_announce = announce
+
+        if selected_announce != None:
+            self.announce_stream.remove(selected_announce)
 
     def display_name(self, source_hash):
         if source_hash in self.directory_entries:
@@ -94,6 +132,12 @@ class Directory:
     def remember(self, entry):
         self.directory_entries[entry.source_hash] = entry
 
+        identity = RNS.Identity.recall(entry.source_hash)
+        associated_node = RNS.Destination.hash_from_name_and_identity("nomadnetwork.node", identity)
+        if associated_node in self.directory_entries:
+            node_entry = self.directory_entries[associated_node]
+            node_entry.trust_level = entry.trust_level
+
     def forget(self, source_hash):
         if source_hash in self.directory_entries:
             self.directory_entries.pop(source_hash)

nomadnet/Node.py

@@ -0,0 +1,145 @@
+import os
+import RNS
+import time
+import threading
+import RNS.vendor.umsgpack as msgpack
+
+class Node:
+    JOB_INTERVAL = 5
+
+    def __init__(self, app):
+        RNS.log("Nomad Network Node starting...", RNS.LOG_VERBOSE)
+        self.app = app
+        self.identity = self.app.identity
+        self.destination = RNS.Destination(self.identity, RNS.Destination.IN, RNS.Destination.SINGLE, "nomadnetwork", "node")
+        self.last_announce = None
+        self.announce_interval = self.app.node_announce_interval
+        self.job_interval = Node.JOB_INTERVAL
+        self.should_run_jobs = True
+        self.app_data = None
+        self.name = self.app.node_name
+
+        self.register_pages()
+        self.register_files()
+
+        if self.name == None:
+            self.name = self.app.peer_settings["display_name"]+"'s Node"
+
+        RNS.log("Node \""+self.name+"\" ready for incoming connections on "+RNS.prettyhexrep(self.destination.hash), RNS.LOG_VERBOSE)
+
+        if self.app.node_announce_at_start:
+            self.announce()
+
+
+    def register_pages(self):
+        self.servedpages = []
+        self.scan_pages(self.app.pagespath)
+
+        if not self.app.pagespath+"index.mu" in self.servedpages:
+            self.destination.register_request_handler(
+                "/page/index.mu",
+                response_generator = self.serve_default_index,
+                allow = RNS.Destination.ALLOW_ALL
+            )
+
+        for page in self.servedpages:
+            request_path = "/page"+page.replace(self.app.pagespath, "")
+            self.destination.register_request_handler(
+                request_path,
+                response_generator = self.serve_page,
+                allow = RNS.Destination.ALLOW_ALL
+            )
+
+    def register_files(self):
+        self.servedfiles = []
+        self.scan_files(self.app.filespath)
+
+        for file in self.servedfiles:
+            request_path = "/file"+file.replace(self.app.filespath, "")
+            self.destination.register_request_handler(
+                request_path,
+                response_generator = self.serve_file,
+                allow = RNS.Destination.ALLOW_ALL
+            )
+
+    def scan_pages(self, base_path):
+        files = [file for file in os.listdir(base_path) if os.path.isfile(os.path.join(base_path, file)) and file[:1] != "."]
+        directories = [file for file in os.listdir(base_path) if os.path.isdir(os.path.join(base_path, file)) and file[:1] != "."]
+
+        for file in files:
+            self.servedpages.append(base_path+"/"+file)
+
+        for directory in directories:
+            self.scan_pages(base_path+"/"+directory)
+
+    def scan_files(self, base_path):
+        files = [file for file in os.listdir(base_path) if os.path.isfile(os.path.join(base_path, file)) and file[:1] != "."]
+        directories = [file for file in os.listdir(base_path) if os.path.isdir(os.path.join(base_path, file)) and file[:1] != "."]
+
+        for file in files:
+            self.servedfiles.append(base_path+"/"+file)
+
+        for directory in directories:
+            self.scan_files(base_path+"/"+directory)
+
+    def serve_page(self, path, data, request_id, remote_identity, requested_at):
+        RNS.log("Page request "+RNS.prettyhexrep(request_id)+" for: "+str(path), RNS.LOG_VERBOSE)
+        file_path = path.replace("/page", self.app.pagespath, 1)
+        try:
+            RNS.log("Serving page: "+file_path, RNS.LOG_VERBOSE)
+            fh = open(file_path, "rb")
+            response_data = fh.read()
+            fh.close()
+            return response_data
+
+        except Exception as e:
+            RNS.log("Error occurred while handling request "+RNS.prettyhexrep(request_id)+" for: "+str(path), RNS.LOG_ERROR)
+            RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
+            return None
+
+    # TODO: Improve file handling, this will be slow for large files
+    def serve_file(self, path, data, request_id, remote_identity, requested_at):
+        RNS.log("File request "+RNS.prettyhexrep(request_id)+" for: "+str(path), RNS.LOG_VERBOSE)
+        file_path = path.replace("/file", self.app.filespath, 1)
+        file_name = path.replace("/file/", "", 1)
+        try:
+            RNS.log("Serving file: "+file_path, RNS.LOG_VERBOSE)
+            fh = open(file_path, "rb")
+            file_data = fh.read()
+            fh.close()
+            return [file_name, file_data]
+
+        except Exception as e:
+            RNS.log("Error occurred while handling request "+RNS.prettyhexrep(request_id)+" for: "+str(path), RNS.LOG_ERROR)
+            RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
+            return None
+
+    def serve_default_index(self, path, data, request_id, remote_identity, requested_at):
+        RNS.log("Serving default index for request "+RNS.prettyhexrep(request_id)+" for: "+str(path), RNS.LOG_VERBOSE)
+        return DEFAULT_INDEX.encode("utf-8")
+
+    def announce(self):
+        self.app_data = self.name.encode("utf-8")
+        self.last_announce = time.time()
+        self.destination.announce(app_data=self.app_data)
+
+    def __jobs(self):
+        while self.should_run_jobs:
+            now = time.time()
+            
+            if now > self.last_announce + self.announce_interval:
+                self.announce()
+
+            time.sleep(self.job_interval)
+
+    def peer_connected(link):
+        RNS.log("Peer connected to "+str(self.destination), RNS.LOG_INFO)
+        link.set_link_closed_callback(self.peer_disconnected)
+
+
+DEFAULT_INDEX = '''>Default Home Page
+
+This node is serving pages, but the home page file (index.mu) was not found in the page storage directory. This is an auto-generated placeholder.
+
+If you are the node operator, you can define your own home page by creating a file named `*index.mu`* in the page storage directory.
+'''

nomadnet/NomadNetworkApp.py

@@ -51,6 +51,13 @@ class NomadNetworkApp:
         self.directorypath     = self.configdir+"/storage/directory"
         self.peersettingspath  = self.configdir+"/storage/peersettings"
 
+        self.pagespath         = self.configdir+"/storage/pages"
+        self.filespath         = self.configdir+"/storage/files"
+
+        self.downloads_path    = os.path.expanduser("~/Downloads")
+
+        self.firstrun          = False
+
         if not os.path.isdir(self.storagepath):
             os.makedirs(self.storagepath)
 
@@ -63,6 +70,12 @@ class NomadNetworkApp:
         if not os.path.isdir(self.conversationpath):
             os.makedirs(self.conversationpath)
 
+        if not os.path.isdir(self.pagespath):
+            os.makedirs(self.pagespath)
+
+        if not os.path.isdir(self.filespath):
+            os.makedirs(self.filespath)
+
         if os.path.isfile(self.configpath):
             try:
                 self.config = ConfigObj(self.configpath)
@@ -80,7 +93,7 @@ class NomadNetworkApp:
         else:
             RNS.log("Could not load config file, creating default configuration file...")
             self.createDefaultConfig()
-
+            self.firstrun = True
 
         if os.path.isfile(self.identitypath):
             try:
@@ -118,7 +131,7 @@ class NomadNetworkApp:
             try:
                 RNS.log("No peer settings file found, creating new...")
                 self.peer_settings = {
-                    "display_name": "",
+                    "display_name": "Anonymous Peer",
                     "announce_interval": None,
                     "last_announce": None,
                 }
@@ -145,11 +158,17 @@ class NomadNetworkApp:
             app_data=None
         )
 
-        RNS.Transport.register_announce_handler(nomadnet.Conversation)
-
         RNS.log("LXMF Router ready to receive on: "+RNS.prettyhexrep(self.lxmf_destination.hash))
 
-        self.directory = nomadnet.Directory.Directory(self)
+        self.directory = nomadnet.Directory(self)
+
+        if self.enable_node:
+            self.node = nomadnet.Node(self)
+        else:
+            self.node = None
+
+        RNS.Transport.register_announce_handler(nomadnet.Conversation)
+        RNS.Transport.register_announce_handler(nomadnet.Directory)
 
         nomadnet.ui.spawn(self.uimode)
 
@@ -190,6 +209,18 @@ class NomadNetworkApp:
     def conversations(self):
         return nomadnet.Conversation.conversation_list(self)
 
+    def conversation_is_unread(self, source_hash):
+        if bytes.fromhex(source_hash) in nomadnet.Conversation.unread_conversations:
+            return True
+        else:
+            return False
+
+    def mark_conversation_read(self, source_hash):
+        if bytes.fromhex(source_hash) in nomadnet.Conversation.unread_conversations:
+            nomadnet.Conversation.unread_conversations.pop(bytes.fromhex(source_hash))
+            if os.path.isfile(self.conversationpath + "/" + source_hash + "/unread"):
+                os.unlink(self.conversationpath + "/" + source_hash + "/unread")
+
     def createDefaultConfig(self):
         self.config = ConfigObj(__default_nomadnet_config__)
         self.config.filename = self.configpath
@@ -227,6 +258,10 @@ class NomadNetworkApp:
                     value = self.config["client"].as_bool(option)
                     self.enable_client = value
 
+                if option == "downloads_path":
+                    value = self.config["client"]["downloads_path"]
+                    self.downloads_path = os.path.expanduser(value)
+
                 if option == "user_interface":
                     value = value.lower()
                     if value == "none":
@@ -295,13 +330,40 @@ class NomadNetworkApp:
                         self.uimode = nomadnet.ui.UI_WEB
 
         if "node" in self.config:
-            for option in self.config["node"]:
+            if not "enable_node" in self.config["node"]:
-                value = self.config["node"][option]
+                self.enable_node = False
+            else:
+                self.enable_node = self.config["node"].as_bool("enable_node")
 
-                if option == "enable_node":
+            if not "node_name" in self.config["node"]:
-                    value = self.config["node"].as_bool(option)
+                self.node_name = None
-                    self.enable_node = value
+            else:
+                value = self.config["node"]["node_name"]
+                if value.lower() == "none":
+                    self.node_name = None
+                else:
+                    self.node_name = self.config["node"]["node_name"]
 
+            if not "announce_at_start" in self.config["node"]:
+                self.node_announce_at_start = False
+            else:
+                self.node_announce_at_start = self.config["node"]["announce_at_start"]
+
+            if not "announce_interval" in self.config["node"]:
+                self.node_announce_interval = 720
+            else:
+                value = self.config["node"].as_int("announce_interval")
+                if value < 1:
+                    value = 1
+                self.node_announce_interval = value
+
+            if "pages_path" in self.config["node"]:
+                self.pagespath = self.config["node"]["pages_path"]
+
+            if "files_path" in self.config["node"]:
+                self.filespath = self.config["node"]["files_path"]
+
+              
     @staticmethod
     def get_shared_instance():
         if NomadNetworkApp._shared_instance != None:
@@ -337,6 +399,7 @@ destination = file
 
 enable_client = yes
 user_interface = text
+downloads_path = ~/Downloads
 
 [textui]
 
@@ -347,15 +410,15 @@ intro_time = 1
 # valid colormodes are:
 # monochrome, 16, 88, 256 and 24bit
 #
-# The default is a conservative 16 colors,
+# The default is a conservative 256 colors.
-# but 256 colors can probably be used on
+# If your terminal does not support this,
-# most terminals. Some terminals support
+# you can lower it. Some terminals support
 # 24 bit color.
 
 # colormode = monochrome
-colormode = 16
+# colormode = 16
 # colormode = 88
-# colormode = 256
+colormode = 256
 # colormode = 24bit
 
 # By default, unicode glyphs are used. If
@@ -385,6 +448,23 @@ hide_guide = no
 
 [node]
 
+# Whether to enable node hosting
+
 enable_node = no
 
+# The node name will be visible to other
+# peers on the network, and included in
+# announces.
+
+node_name = None
+
+# Automatic announce interval in minutes.
+# 12 hours by default.
+
+announce_interval = 720
+
+# Whether to announce when the node starts
+
+announce_at_start = No
+
 '''.splitlines()

nomadnet/__init__.py

@@ -3,6 +3,8 @@ import glob
 
 from .NomadNetworkApp import NomadNetworkApp
 from .Conversation import Conversation
+from .Directory import Directory
+from .Node import Node
 from .ui import *
 
 

nomadnet/_version.py

@@ -1 +1 @@
-__version__ = "0.0.6"
+__version__ = "0.0.8"

nomadnet/ui/TextUI.py

@@ -18,30 +18,34 @@ THEMES = {
     THEME_DARK: {
         "urwid_theme": [
             # Style name                    # 16-color style                        # Monochrome style          # 88, 256 and true-color style
-            ('heading',                     'light gray,underline', 'default',      'underline',                'g93,underline', 'default'),
+            ("heading",                     "light gray,underline", "default",      "underline",                "g93,underline", "default"),
-            ('menubar',                     'black', 'light gray',                  'standout',                 '#111', '#bbb'),
+            ("menubar",                     "black", "light gray",                  "standout",                 "#111", "#bbb"),
-            ('scrollbar',                   'black', 'light gray',                  'standout',                 '#444', 'default'),
+            ("scrollbar",                   "light gray", "default",                "standout",                 "#444", "default"),
-            ('shortcutbar',                 'black', 'light gray',                  'standout',                 '#111', '#bbb'),
+            ("shortcutbar",                 "black", "light gray",                  "standout",                 "#111", "#bbb"),
-            ('body_text',                   'white', 'default',                     'default',                  '#ddd', 'default'),
+            ("body_text",                   "white", "default",                     "default",                  "#ddd", "default"),
-            ('error_text',                  'dark red', 'default',                  'default',                  'dark red', 'default'),
+            ("error_text",                  "dark red", "default",                  "default",                  "dark red", "default"),
-            ('warning_text',                'yellow', 'default',                    'default',                  '#ba4', 'default'),
+            ("warning_text",                "yellow", "default",                    "default",                  "#ba4", "default"),
-            ('inactive_text',               'dark gray', 'default',                 'default',                  'dark gray', 'default'),
+            ("inactive_text",               "dark gray", "default",                 "default",                  "dark gray", "default"),
-            ('buttons',                     'light green,bold', 'default',          'default',                  '#00a533', 'default'),
+            ("buttons",                     "light green,bold", "default",          "default",                  "#00a533", "default"),
-            ('msg_editor',                  'black', 'light cyan',                  'standout',                 '#111', '#0bb'),
+            ("msg_editor",                  "black", "light cyan",                  "standout",                 "#111", "#0bb"),
-            ("msg_header_ok",               'black', 'light green',                 'standout',                 '#111', '#6b2'),
+            ("msg_header_ok",               "black", "light green",                 "standout",                 "#111", "#6b2"),
-            ("msg_header_caution",          'black', 'yellow',                      'standout',                 '#111', '#fd3'),
+            ("msg_header_caution",          "black", "yellow",                      "standout",                 "#111", "#fd3"),
-            ("msg_header_sent",             'black', 'light gray',                  'standout',                 '#111', '#ddd'),
+            ("msg_header_sent",             "black", "light gray",                  "standout",                 "#111", "#ddd"),
-            ("msg_header_delivered",        'black', 'light blue',                  'standout',                 '#111', '#28b'),
+            ("msg_header_delivered",        "black", "light blue",                  "standout",                 "#111", "#28b"),
-            ("msg_header_failed",           'black', 'dark gray',                   'standout',                 'black', 'dark gray'),
+            ("msg_header_failed",           "black", "dark gray",                   "standout",                 "#000", "#777"),
-            ("msg_warning_untrusted",       'black', 'dark red',                    'standout',                 '#111', 'dark red'),
+            ("msg_warning_untrusted",       "black", "dark red",                    "standout",                 "#111", "dark red"),
-            ("list_focus",                  "black", "light gray",                  "standout",                 "#111", "#bbb"),
+            ("list_focus",                  "black", "light gray",                  "standout",                 "#111", "#aaa"),
-            ("list_off_focus",              "black", "dark gray",                   "standout",                 "#111", "dark gray"),
+            ("list_off_focus",              "black", "dark gray",                   "standout",                 "#111", "#777"),
-            ("list_trusted",                "light green", "default",               "default",                  "#6b2", "default"),
+            ("list_trusted",                "dark green", "default",                "default",                  "#6b2", "default"),
-            ("list_focus_trusted",          "black", "light gray",                  "standout",                 "#180", "#bbb"),
+            ("list_focus_trusted",          "black", "light gray",                  "standout",                 "#150", "#aaa"),
-            ("list_unknown",                "dark gray", "default",                 "default",                  "light gray", "default"),
+            ("list_unknown",                "dark gray", "default",                 "default",                  "#bbb", "default"),
-            ("list_normal",                 "dark gray", "default",                 "default",                  "light gray", "default"),
+            ("list_normal",                 "dark gray", "default",                 "default",                  "#bbb", "default"),
-            ("list_untrusted",              "dark red", "default",                  "default",                  "dark red", "default"),
+            ("list_untrusted",              "dark red", "default",                  "default",                  "#a22", "default"),
-            ("list_focus_untrusted",        "black", "light gray",                  "standout",                 "#810", "#bbb"),
+            ("list_focus_untrusted",        "black", "light gray",                  "standout",                 "#810", "#aaa"),
+            ("topic_list_normal",           "white", "default",                     "default",                  "#ddd", "default"),
+            ("browser_controls",            "light gray", "default",                "default",                  "#bbb", "default"),
+            ("progress_full",               "black", "light gray",                  "standout",                 "#111", "#bbb"),
+            ("progress_empty",              "light gray", "default",                "default",                  "#ddd", "default"),
         ],        
     }
 }
@@ -65,7 +69,12 @@ GLYPHS = {
     ("arrow_d",         "\\/",       "\u2193",      "\u2193"),
     ("warning",         "!",         "\u26a0",      "\uf12a"),
     ("info",            "i",         "\u2139",      "\ufb4d"),
+    ("unread",          "N",         "\u2709",      "\uf003 "),
     ("divider1",        "-",         "\u2504",      "\u2504"),
+    ("peer",            "P",         "\U0001F464",  "\uf415"),
+    ("node",            "N",         "\U0001F5A5",  "\uf502"),
+    ("page",            "",          "\U0001F5B9",  "\uf719 "),
+    ("speed",           "",          "\U0001F5B9",  "\uf9c4"),
     ("decoration_menu", "",          "",            " \uf93a"),
 }
 
@@ -124,13 +133,14 @@ class TextUI:
         # TODO: Probably remove this at some point when better terminal
         # color capability detection has been implemented
         if colormode > 16:
-            RNS.log("Starting Text UI in "+str(colormode)+" color mode. If no UI appears, try adjusting your color settings in "+str(self.app.configdir)+"/config", RNS.LOG_INFO)
+            RNS.log("Starting Text UI in "+str(colormode)+" color mode. If no UI appears, try adjusting your color settings in "+str(self.app.configdir)+"/config", RNS.LOG_INFO, _override_destination = True)
         
         self.set_colormode(colormode)
 
         self.loop.run()
 
     def set_colormode(self, colormode):
+        self.colormode = colormode
         self.screen.set_terminal_properties(colormode)
         self.screen.reset_default_terminal_palette()
 

nomadnet/ui/textui/Browser.py

@@ -0,0 +1,571 @@
+import RNS
+import os
+import time
+import urwid
+import nomadnet
+import threading
+from .MicronParser import markup_to_attrmaps
+from nomadnet.vendor.Scrollable import *
+
+class BrowserFrame(urwid.Frame):
+    def keypress(self, size, key):
+        if key == "ctrl w":
+            self.delegate.disconnect()
+        elif self.get_focus() == "body":
+            return super(BrowserFrame, self).keypress(size, key)
+            # if key == "up" and self.delegate.messagelist.top_is_visible:
+            #     nomadnet.NomadNetworkApp.get_shared_instance().ui.main_display.frame.set_focus("header")
+            # elif key == "down" and self.delegate.messagelist.bottom_is_visible:
+            #     self.set_focus("footer")
+            # else:
+            #     return super(ConversationFrame, self).keypress(size, key)
+        else:
+            return super(BrowserFrame, self).keypress(size, key)
+
+class Browser:
+    DEFAULT_PATH       = "/page/index.mu"
+    DEFAULT_TIMEOUT    = 10
+
+    NO_PATH            = 0x00
+    PATH_REQUESTED     = 0x01
+    ESTABLISHING_LINK  = 0x02
+    LINK_TIMEOUT       = 0x03
+    LINK_ESTABLISHED   = 0x04
+    REQUESTING         = 0x05
+    REQUEST_SENT       = 0x06
+    REQUEST_FAILED     = 0x07
+    REQUEST_TIMEOUT    = 0x08
+    RECEIVING_RESPONSE = 0x09
+    DISCONECTED        = 0xFE
+    DONE               = 0xFF
+
+    def __init__(self, app, app_name, aspects, destination_hash = None, path = None, auth_identity = None, delegate = None):
+        self.app = app
+        self.g = self.app.ui.glyphs
+        self.delegate = delegate
+        self.app_name = app_name
+        self.aspects = aspects
+        self.destination_hash = destination_hash
+        self.path = path
+        self.timeout = Browser.DEFAULT_TIMEOUT
+        self.last_keypress = None
+
+        self.link = None
+        self.status = Browser.DISCONECTED
+        self.response_progress = 0
+        self.response_size = None
+        self.response_transfer_size = None
+        self.saved_file_name = None
+        self.page_data = None
+        self.displayed_page_data = None
+        self.auth_identity = auth_identity
+        self.display_widget = None
+        self.link_status_showing = False
+        self.link_target = None
+        self.frame = None
+        self.attr_maps = []
+        self.build_display()
+
+        if self.path == None:
+            self.path = Browser.DEFAULT_PATH
+
+        if self.destination_hash != None:
+            self.load_page()
+
+    def current_url(self):
+        if self.destination_hash == None:
+            return ""
+        else:
+            if self.path == None:
+                path = ""
+            else:
+                path = self.path
+            return RNS.hexrep(self.destination_hash, delimit=False)+":"+path
+
+
+    def marked_link(self, link_target):
+        self.link_target = link_target
+        self.app.ui.loop.set_alarm_in(0.1, self.marked_link_job)
+
+    def marked_link_job(self, sender, event):
+        link_target = self.link_target
+
+        if link_target == None:
+            if self.link_status_showing:
+                self.browser_footer = self.make_status_widget()
+                self.frame.contents["footer"] = (self.browser_footer, self.frame.options())
+                self.link_status_showing = False
+        else:
+            self.link_status_showing = True
+            self.browser_footer = urwid.AttrMap(urwid.Pile([urwid.Divider(self.g["divider1"]), urwid.Text("Link to: "+str(link_target))]), "browser_controls")
+            self.frame.contents["footer"] = (self.browser_footer, self.frame.options())
+
+    def handle_link(self, link_target):
+        if self.status >= Browser.DISCONECTED:
+            RNS.log("Browser handling link to: "+str(link_target), RNS.LOG_DEBUG)
+            try:
+                self.retrieve_url(link_target)
+            except Exception as e:
+                self.browser_footer = urwid.Text("Could not open link: "+str(e))
+                self.frame.contents["footer"] = (self.browser_footer, self.frame.options())
+        else:
+            RNS.log("Browser aleady hadling link, cannot handle link to: "+str(link_target), RNS.LOG_DEBUG)
+
+
+
+    def micron_released_focus(self):
+        if self.delegate != None:
+            self.delegate.focus_lists()
+
+    def build_display(self):
+        self.browser_header = urwid.Text("")
+        self.browser_footer = urwid.Text("")
+
+        self.browser_body = urwid.Filler(urwid.Text("Disconnected\n"+self.g["arrow_l"]+"  "+self.g["arrow_r"], align="center"), "middle")
+
+        self.frame = BrowserFrame(self.browser_body, header=self.browser_header, footer=self.browser_footer)
+        self.frame.delegate = self
+        self.linebox = urwid.LineBox(self.frame, title="Remote Node")
+        self.display_widget = urwid.AttrMap(self.linebox, "inactive_text")
+
+    def make_status_widget(self):
+        if self.response_progress > 0:
+            pb = ResponseProgressBar("progress_empty" , "progress_full", current=self.response_progress, done=1.0, satt=None)
+            widget = urwid.Pile([urwid.Divider(self.g["divider1"]), pb])
+        else:
+            widget = urwid.Pile([urwid.Divider(self.g["divider1"]), urwid.Text(self.status_text())])
+
+        return urwid.AttrMap(widget, "browser_controls")
+
+    def make_control_widget(self):
+        return urwid.AttrMap(urwid.Pile([urwid.Text(self.g["node"]+" "+self.current_url()), urwid.Divider(self.g["divider1"])]), "browser_controls")
+
+    def make_request_failed_widget(self):
+        def back_action(sender):
+            self.status = Browser.DONE
+            self.destination_hash = self.previous_destination_hash
+            self.path = self.previous_path
+            self.update_display()
+
+        columns = urwid.Columns([
+            ("weight", 0.5, urwid.Text(" ")),
+            (8, urwid.Button("Back", on_press=back_action)),
+            ("weight", 0.5, urwid.Text(" "))
+        ])
+
+        if len(self.attr_maps) > 0:
+            pile = urwid.Pile([
+                    urwid.Text("!\n\n"+self.status_text()+"\n", align="center"),
+                    columns
+            ])
+        else:
+            pile = urwid.Pile([
+                    urwid.Text("!\n\n"+self.status_text(), align="center")
+            ])
+
+        return urwid.Filler(pile, "middle")
+    
+    def update_display(self):
+        if self.status == Browser.DISCONECTED:
+            self.display_widget.set_attr_map({None: "inactive_text"})
+            self.browser_body = urwid.Filler(urwid.Text("Disconnected\n"+self.g["arrow_l"]+"  "+self.g["arrow_r"], align="center"), "middle")
+            self.browser_footer = urwid.Text("")
+            self.browser_header = urwid.Text("")
+            self.linebox.set_title("Remote Node")
+        else:
+            self.display_widget.set_attr_map({None: "body_text"})
+            self.browser_header = self.make_control_widget()
+            self.linebox.set_title(self.app.directory.simplest_display_str(self.destination_hash))
+
+            if self.status == Browser.DONE:
+                self.browser_footer = self.make_status_widget()
+                self.update_page_display()
+            
+            elif self.status == Browser.LINK_TIMEOUT:
+                self.browser_body = self.make_request_failed_widget()
+                self.browser_footer = urwid.Text("")
+            
+            elif self.status <= Browser.REQUEST_SENT:
+                if len(self.attr_maps) == 0:
+                    self.browser_body = urwid.Filler(urwid.Text("Retrieving\n["+self.current_url()+"]", align="center"), "middle")
+
+                self.browser_footer = self.make_status_widget()
+            
+            elif self.status == Browser.REQUEST_FAILED:
+                self.browser_body = self.make_request_failed_widget()
+                self.browser_footer = urwid.Text("")
+            
+            elif self.status == Browser.REQUEST_TIMEOUT:
+                self.browser_body = self.make_request_failed_widget()
+                self.browser_footer = urwid.Text("")
+            
+            else:
+                pass
+
+        self.frame.contents["body"] = (self.browser_body, self.frame.options())
+        self.frame.contents["header"] = (self.browser_header, self.frame.options())
+        self.frame.contents["footer"] = (self.browser_footer, self.frame.options())
+
+    def update_page_display(self):
+        pile = urwid.Pile(self.attr_maps)
+        self.browser_body = urwid.AttrMap(ScrollBar(Scrollable(pile, force_forward_keypress=True), thumb_char="\u2503", trough_char=" "), "scrollbar")
+
+    def identify(self):
+        if self.link != None:
+            if self.link.status == RNS.Link.ACTIVE:
+                self.link.identify(self.auth_identity)
+
+
+    def disconnect(self):
+        if self.link != None:
+            self.link.teardown()
+        
+        self.attr_maps = []
+        self.status = Browser.DISCONECTED
+        self.response_progress = 0
+        self.response_size = None
+        self.response_transfer_size = None
+
+        self.update_display()
+
+
+    def retrieve_url(self, url):
+        self.previous_destination_hash = self.destination_hash
+        self.previous_path = self.path
+
+        destination_hash = None
+        path = None
+
+        components = url.split(":")
+        if len(components) == 1:
+            if len(components[0]) == 20:
+                try:
+                    destination_hash = bytes.fromhex(components[0])
+                except Exception as e:
+                    raise ValueError("Malformed URL")
+                path = Browser.DEFAULT_PATH
+            else:
+                raise ValueError("Malformed URL")
+        elif len(components) == 2:
+            if len(components[0]) == 20:
+                try:
+                    destination_hash = bytes.fromhex(components[0])
+                except Exception as e:
+                    raise ValueError("Malformed URL")
+                path = components[1]
+                if len(path) == 0:
+                    path = Browser.DEFAULT_PATH
+            else:
+                if len(components[0]) == 0:
+                    if self.destination_hash != None:
+                        destination_hash = self.destination_hash
+                        path = components[1]
+                        if len(path) == 0:
+                            path = Browser.DEFAULT_PATH
+                    else:
+                        raise ValueError("Malformed URL")
+                else:
+                        raise ValueError("Malformed URL")
+        else:
+            raise ValueError("Malformed URL")
+
+        if destination_hash != None and path != None:
+            if path.startswith("/file/"):
+                if destination_hash == self.destination_hash:
+                    self.download_file(destination_hash, path)
+                else:
+                    RNS.log("Cannot request file download from a node that is not currently connected.", RNS.LOG_ERROR)
+                    RNS.log("The requested URL was: "+str(url), RNS.LOG_ERROR)
+            else:
+                self.set_destination_hash(destination_hash)
+                self.set_path(path)
+                self.load_page()
+
+    def set_destination_hash(self, destination_hash):
+        if len(destination_hash) == RNS.Identity.TRUNCATED_HASHLENGTH//8:
+            self.destination_hash = destination_hash
+            return True
+        else:
+            return False
+
+
+    def set_path(self, path):
+        self.path = path
+
+
+    def set_timeout(self, timeout):
+        self.timeout = timeout
+
+
+    def download_file(self, destination_hash, path):
+        if self.link != None and self.link.destination.hash == self.destination_hash:
+            # Send the request
+            self.status = Browser.REQUESTING
+            self.response_progress = 0
+            self.response_size = None
+            self.response_transfer_size = None
+            self.saved_file_name = None
+
+            self.update_display()
+            receipt = self.link.request(
+                path,
+                data = None,
+                response_callback = self.file_received,
+                failed_callback = self.request_failed,
+                progress_callback = self.response_progressed
+            )
+
+            if receipt:
+                self.last_request_receipt = receipt
+                self.last_request_id = receipt.request_id
+                self.status = Browser.REQUEST_SENT
+                self.update_display()
+            else:
+                self.link.teardown()
+
+
+    def load_page(self):
+        load_thread = threading.Thread(target=self.__load)
+        load_thread.setDaemon(True)
+        load_thread.start()
+
+
+    def __load(self):
+        # If an established link exists, but it doesn't match the target
+        # destination, we close and clear it.
+        if self.link != None and self.link.destination.hash != self.destination_hash:
+            self.link.teardown()
+            self.link = None
+
+        # If no link to the destination exists, we create one.
+        if self.link == None:
+            if not RNS.Transport.has_path(self.destination_hash):
+                self.status = Browser.NO_PATH
+                self.update_display()
+
+                RNS.Transport.request_path(self.destination_hash)
+                self.status = Browser.PATH_REQUESTED
+                self.update_display()
+
+                pr_time = time.time()
+                while not RNS.Transport.has_path(self.destination_hash):
+                    now = time.time()
+                    if now > pr_time+self.timeout:
+                        self.request_timeout()
+                        return
+
+                    time.sleep(0.25)
+
+            self.status = Browser.ESTABLISHING_LINK
+            self.update_display()
+
+            identity = RNS.Identity.recall(self.destination_hash)
+            destination = RNS.Destination(
+                identity,
+                RNS.Destination.OUT,
+                RNS.Destination.SINGLE,
+                self.app_name,
+                self.aspects
+            )
+
+            self.link = RNS.Link(destination, established_callback = self.link_established, closed_callback = self.link_closed)
+
+            while self.status == Browser.ESTABLISHING_LINK:
+                time.sleep(0.1)
+
+            if self.status != Browser.LINK_ESTABLISHED:
+                return
+
+            self.update_display()
+
+        # Send the request
+        self.status = Browser.REQUESTING
+        self.response_progress = 0
+        self.response_size = None
+        self.response_transfer_size = None
+        self.saved_file_name = None
+
+
+        self.update_display()
+        receipt = self.link.request(
+            self.path,
+            data = None,
+            response_callback = self.response_received,
+            failed_callback = self.request_failed,
+            progress_callback = self.response_progressed
+        )
+
+        if receipt:
+            self.last_request_receipt = receipt
+            self.last_request_id = receipt.request_id
+            self.status = Browser.REQUEST_SENT
+            self.update_display()
+        else:
+            self.link.teardown()
+
+
+
+    def link_established(self, link):
+        self.status = Browser.LINK_ESTABLISHED
+
+
+    def link_closed(self, link):
+        if self.status == Browser.DISCONECTED or self.status == Browser.DONE:
+            self.link = None
+        elif self.status == Browser.ESTABLISHING_LINK:
+            self.link_establishment_timeout()
+        else:
+            self.link = None
+            self.status = Browser.REQUEST_FAILED
+        self.update_display()
+
+    def link_establishment_timeout(self):
+        self.status = Browser.LINK_TIMEOUT
+        self.response_progress = 0
+        self.response_size = None
+        self.response_transfer_size = None
+        self.link = None
+
+        self.update_display()
+
+
+    def response_received(self, request_receipt):
+        try:
+            self.status = Browser.DONE
+            self.page_data = request_receipt.response
+            self.markup = self.page_data.decode("utf-8")
+            self.attr_maps = markup_to_attrmaps(self.markup, url_delegate=self)
+            self.response_progress = 0
+
+            self.update_display()
+        except Exception as e:
+            RNS.log("An error occurred while handling response. The contained exception was: "+str(e))
+
+
+    def file_received(self, request_receipt):
+        try:
+            file_name = request_receipt.response[0]
+            file_data = request_receipt.response[1]
+            file_destination = self.app.downloads_path+"/"+file_name
+            
+            counter = 0
+            while os.path.isfile(file_destination):
+                counter += 1
+                file_destination = self.app.downloads_path+"/"+file_name+"."+str(counter)
+
+            fh = open(file_destination, "wb")
+            fh.write(file_data)
+            fh.close()
+
+            self.saved_file_name = file_destination.replace(self.app.downloads_path+"/", "", 1)
+            self.status = Browser.DONE
+            self.response_progress = 0
+
+            self.update_display()
+        except Exception as e:
+            RNS.log("An error occurred while handling file response. The contained exception was: "+str(e))
+
+    
+    def request_failed(self, request_receipt=None):
+        if request_receipt != None:
+            if request_receipt.request_id == self.last_request_id:
+                self.status = Browser.REQUEST_FAILED
+                self.response_progress = 0
+                self.response_size = None
+                self.response_transfer_size = None
+
+                self.update_display()
+        else:
+            self.status = Browser.REQUEST_FAILED
+            self.response_progress = 0
+            self.response_size = None
+            self.response_transfer_size = None
+
+            self.update_display()
+
+
+    def request_timeout(self, request_receipt=None):
+        self.status = Browser.REQUEST_TIMEOUT
+        self.response_progress = 0
+        self.response_size = None
+        self.response_transfer_size = None
+
+        self.update_display()
+
+
+    def response_progressed(self, request_receipt):
+        self.response_progress      = request_receipt.progress
+        self.response_time          = request_receipt.get_response_time()
+        self.response_size          = request_receipt.response_size
+        self.response_transfer_size = request_receipt.response_transfer_size
+        self.update_display()
+
+
+    def status_text(self):
+        if self.status == Browser.DONE and self.response_transfer_size != None:
+            if self.response_time != None:
+                response_time_str = "{:.2f}".format(self.response_time)
+            else:
+                response_time_str = "None"
+
+            stats_string = "  "+self.g["page"]+size_str(self.response_size)
+            stats_string += "   "+self.g["arrow_d"]+size_str(self.response_transfer_size)+" in "+response_time_str
+            stats_string += "s   "+self.g["speed"]+size_str(self.response_transfer_size/self.response_time, suffix="b")+"/s"
+        else:
+            stats_string = ""
+
+        if self.status == Browser.NO_PATH:
+            return "No path to destination known"
+        elif self.status == Browser.PATH_REQUESTED:
+            return "Path requested, waiting for path..."
+        elif self.status == Browser.ESTABLISHING_LINK:
+            return "Establishing link..."
+        elif self.status == Browser.LINK_TIMEOUT:
+            return "Link establishment timed out"
+        elif self.status == Browser.LINK_ESTABLISHED:
+            return "Link established"
+        elif self.status == Browser.REQUESTING:
+            return "Sending request..."
+        elif self.status == Browser.REQUEST_SENT:
+            return "Request sent, awaiting response..."
+        elif self.status == Browser.REQUEST_FAILED:
+            return "Request failed"
+        elif self.status == Browser.REQUEST_TIMEOUT:
+            return "Request timed out"
+        elif self.status == Browser.RECEIVING_RESPONSE:
+            return "Receiving response..."
+        elif self.status == Browser.DONE:
+            if self.saved_file_name == None:
+                return "Done"+stats_string
+            else:
+                return "Saved "+str(self.saved_file_name)+stats_string
+        elif self.status == Browser.DISCONECTED:
+            return "Disconnected"
+        else:
+            return "Browser Status Unknown"
+
+
+class ResponseProgressBar(urwid.ProgressBar):
+    def get_text(self):
+        return "Receiving response "+super().get_text()
+
+# A convenience function for printing a human-
+# readable file size
+def size_str(num, suffix='B'):
+    units = ['','K','M','G','T','P','E','Z']
+    last_unit = 'Y'
+
+    if suffix == 'b':
+        num *= 8
+        units = ['','K','M','G','T','P','E','Z']
+        last_unit = 'Y'
+
+    for unit in units:
+        if abs(num) < 1000.0:
+            if unit == "":
+                return "%.0f%s%s" % (num, unit, suffix)
+            else:
+                return "%.2f%s%s" % (num, unit, suffix)
+        num /= 1000.0
+
+    return "%.2f%s%s" % (num, last_unit, suffix)

nomadnet/ui/textui/Config.py

@@ -1,5 +1,6 @@
 import nomadnet
 import urwid
+import platform
 
 class ConfigDisplayShortcuts():
     def __init__(self, app):
@@ -50,6 +51,12 @@ class EditorTerminal(urwid.WidgetWrap):
         self.app = app
         self.parent = parent
         editor_cmd = self.app.config["textui"]["editor"]
+
+        # The "editor" alias is unavailable on Darwin,
+        # so we replace it with nano.
+        if platform.system() == "Darwin" and editor_cmd == "editor":
+            editor_cmd = "nano"
+
         self.term = urwid.Terminal(
             (editor_cmd, self.app.configpath),
             encoding='utf-8',

nomadnet/ui/textui/Conversations.py

@@ -1,4 +1,5 @@
 import RNS
+import os
 import time
 import nomadnet
 import LXMF
@@ -19,7 +20,7 @@ class ConversationDisplayShortcuts():
     def __init__(self, app):
         self.app = app
 
-        self.widget = urwid.AttrMap(urwid.Text("[C-d] Send  [C-k] Clear  [C-w] Close  [C-t] Editor Type  [C-p] Purge"), "shortcutbar")
+        self.widget = urwid.AttrMap(urwid.Text("[C-d] Send  [C-k] Clear Editor  [C-w] Close  [C-t] Editor Type  [C-p] Purge  [C-x] Clear History"), "shortcutbar")
 
 class ConversationsArea(urwid.LineBox):
     def keypress(self, size, key):
@@ -86,7 +87,6 @@ class ConversationsDisplay():
         for conversation in self.app.conversations():
             conversation_list_widgets.append(self.conversation_list_widget(conversation))
 
-        walker = urwid.SimpleFocusListWalker(conversation_list_widgets)
         self.list_widgets = conversation_list_widgets
         self.ilb = IndicativeListBox(
             self.list_widgets,
@@ -316,17 +316,43 @@ class ConversationsDisplay():
             self.ilb.select_item(ilb_position)
         nomadnet.NomadNetworkApp.get_shared_instance().ui.loop.draw_screen()
 
+        if self.currently_displayed_conversation != None:
+            if self.app.conversation_is_unread(self.currently_displayed_conversation):
+                self.app.mark_conversation_read(self.currently_displayed_conversation)
+                try:
+                    if os.path.isfile(self.app.conversationpath + "/" + self.currently_displayed_conversation + "/unread"):
+                        os.unlink(self.app.conversationpath + "/" + self.currently_displayed_conversation + "/unread")
+                except Exception as e:
+                    raise e
+
 
 
 
     def display_conversation(self, sender=None, source_hash=None):
+        if self.currently_displayed_conversation != None:
+            if self.app.conversation_is_unread(self.currently_displayed_conversation):
+                self.app.mark_conversation_read(self.currently_displayed_conversation)
+
         self.currently_displayed_conversation = source_hash
         options = self.widget.options("weight", 1-ConversationsDisplay.list_width)
         self.widget.contents[1] = (self.make_conversation_widget(source_hash), options)
         if source_hash == None:
             self.widget.set_focus_column(0)
         else:
+            if self.app.conversation_is_unread(source_hash):
+                self.app.mark_conversation_read(source_hash)
+                self.update_conversation_list()
+
             self.widget.set_focus_column(1)
+            conversation_position = None
+            index = 0
+            for widget in self.list_widgets:
+                if widget.source_hash == source_hash:
+                    conversation_position = index
+                index += 1
+
+            if conversation_position != None:
+                self.ilb.select_item(conversation_position)
         
 
     def make_conversation_widget(self, source_hash):
@@ -364,6 +390,7 @@ class ConversationsDisplay():
         trust_level  = conversation[2]
         display_name = conversation[1]
         source_hash  = conversation[0]
+        unread       = conversation[4]
 
         g = self.app.ui.glyphs
 
@@ -389,11 +416,17 @@ class ConversationsDisplay():
             focus_style   = "list_focus_untrusted"
 
         display_text = symbol
+
         if display_name != None and display_name != "":
             display_text += " "+display_name
 
         if trust_level != DirectoryEntry.TRUSTED:
             display_text += " <"+source_hash+">"
+        else:
+            if unread:
+                if source_hash != self.currently_displayed_conversation:
+                    display_text += " "+g["unread"]
+
 
         widget = ListEntry(display_text)
         urwid.connect_signal(widget, "click", self.display_conversation, conversation[0])
@@ -544,6 +577,31 @@ class ConversationWidget(urwid.WidgetWrap):
                 
                 urwid.WidgetWrap.__init__(self, self.display_widget)
 
+    def clear_history_dialog(self):
+        def dismiss_dialog(sender):
+            self.dialog_open = False
+            self.conversation_changed(None)
+
+        def confirmed(sender):
+            self.dialog_open = False
+            self.conversation.clear_history()
+            self.conversation_changed(None)
+
+
+        dialog = DialogLineBox(
+            urwid.Pile([
+                urwid.Text("Clear conversation history\n", align="center"),
+                urwid.Columns([("weight", 0.45, urwid.Button("Yes", on_press=confirmed)), ("weight", 0.1, urwid.Text("")), ("weight", 0.45, urwid.Button("No", on_press=dismiss_dialog))])
+            ]), title="?"
+        )
+        dialog.delegate = self
+        bottom = self.messagelist
+
+        overlay = urwid.Overlay(dialog, bottom, align="center", width=34, valign="middle", height="pack", left=2, right=2)
+
+        self.frame.contents["body"] = (overlay, self.frame.options())
+        self.frame.set_focus("body")
+    
     def toggle_editor(self):
         if self.full_editor_active:
             self.frame.contents["footer"] = (self.minimal_editor, None)
@@ -559,7 +617,7 @@ class ConversationWidget(urwid.WidgetWrap):
             if allowed:
                 self.frame.contents["footer"] = (self.minimal_editor, None)
             else:
-                warning = urwid.AttrMap(urwid.Padding(urwid.Text(g["info"]+" You cannot currently communicate with this peer, since it's identity keys are unknown", align="center")), "msg_header_caution")
+                warning = urwid.AttrMap(urwid.Padding(urwid.Text(g["info"]+" You cannot currently communicate with this peer, since it's identity keys are not known", align="center")), "msg_header_caution")
                 self.frame.contents["footer"] = (warning, None)
 
     def toggle_focus_area(self):
@@ -584,6 +642,8 @@ class ConversationWidget(urwid.WidgetWrap):
             self.conversation_changed(None)
         elif key == "ctrl t":
             self.toggle_editor()
+        elif key == "ctrl x":
+            self.clear_history_dialog()
         else:
             return super(ConversationWidget, self).keypress(size, key)
 

nomadnet/ui/textui/Guide.py

@@ -10,7 +10,7 @@ class GuideDisplayShortcuts():
         self.app = app
         g = app.ui.glyphs
 
-        self.widget = urwid.AttrMap(urwid.Text(""), "shortcutbar")
+        self.widget = urwid.AttrMap(urwid.Padding(urwid.Text(""), align="left"), "shortcutbar")
 
 class ListEntry(urwid.Text):
     _selectable = True
@@ -61,41 +61,63 @@ class SelectText(urwid.Text):
         return True
 
 class GuideEntry(urwid.WidgetWrap):
-    def __init__(self, app, reader, topic_name):
+    def __init__(self, app, parent, reader, topic_name):
         self.app = app
+        self.parent = parent
         self.reader = reader
+        self.last_keypress = None
+        self.topic_name = topic_name
         g = self.app.ui.glyphs
 
         widget = ListEntry(topic_name)
-        urwid.connect_signal(widget, "click", self.display_topic, topic_name)
+        urwid.connect_signal(widget, "click", self.display_topic, self.topic_name)
 
-        style = "list_normal"
+        style = "topic_list_normal"
         focus_style = "list_focus"
         self.display_widget = urwid.AttrMap(widget, style, focus_style)
         urwid.WidgetWrap.__init__(self, self.display_widget)
 
     def display_topic(self, event, topic):
         markup = TOPICS[topic]
-        attrmaps = markup_to_attrmaps(markup)
+        attrmaps = markup_to_attrmaps(markup, url_delegate=None)
+
+        topic_position = None
+        index = 0
+        for topic in self.parent.topic_list:
+            widget = topic._original_widget
+            if widget.topic_name == self.topic_name:
+                topic_position = index
+            index += 1
+
+        if topic_position != None:
+            self.parent.ilb.select_item(topic_position)
 
         self.reader.set_content_widgets(attrmaps)
+        self.reader.focus_reader()
+
+
+    def micron_released_focus(self):
+        self.reader.focus_topics()
 
 class TopicList(urwid.WidgetWrap):
     def __init__(self, app, guide_display):
         self.app = app
         g = self.app.ui.glyphs
 
+        self.first_run_entry = GuideEntry(self.app, self, guide_display, "First Run")
+
         self.topic_list = [
-            GuideEntry(self.app, guide_display, "Introduction"),
+            GuideEntry(self.app, self, guide_display, "Introduction"),
-            # GuideEntry(self.app, guide_display, "Conversations"),
+            GuideEntry(self.app, self, guide_display, "Hosting a Node"),
-            GuideEntry(self.app, guide_display, "Markup"),
+            GuideEntry(self.app, self, guide_display, "Markup"),
-            GuideEntry(self.app, guide_display, "First Run"),
+            self.first_run_entry,
-            GuideEntry(self.app, guide_display, "Credits & Licenses"),
+            GuideEntry(self.app, self, guide_display, "Credits & Licenses"),
         ]
 
         self.ilb = IndicativeListBox(
             self.topic_list,
             initialization_is_selection_change=False,
+            highlight_offFocus="list_off_focus"
         )
 
         urwid.WidgetWrap.__init__(self, urwid.LineBox(self.ilb, title="Topics"))
@@ -114,7 +136,7 @@ class GuideDisplay():
         self.app = app
         g = self.app.ui.glyphs
 
-        topic_text = urwid.Text("\nNo topic selected", align="left")
+        topic_text = urwid.Text("\n  No topic selected", align="left")
 
         self.left_area  = TopicList(self.app, self)
         self.right_area = urwid.LineBox(urwid.Filler(topic_text, "top"))
@@ -131,6 +153,10 @@ class GuideDisplay():
         self.shortcuts_display = GuideDisplayShortcuts(self.app)
         self.widget = self.columns
 
+        if self.app.firstrun:
+            entry = self.left_area.first_run_entry
+            entry.display_topic(entry.display_topic, entry.topic_name)
+
     def set_content_widgets(self, new_content):
         options = self.columns.options(width_type="weight", width_amount=1-GuideDisplay.list_width)
         pile = urwid.Pile(new_content)
@@ -142,6 +168,12 @@ class GuideDisplay():
     def shortcuts(self):
         return self.shortcuts_display
 
+    def focus_topics(self):
+        self.columns.focus_position = 0
+
+    def focus_reader(self):
+        self.columns.focus_position = 1
+
 
 TOPIC_INTRODUCTION = '''>Nomad Network
 
@@ -168,7 +200,7 @@ A `*peer`* refers to another Nomad Network client, which will generally be opera
 
 An `*announce`* can be sent by any peer on the network, which will notify other peers of its existence, and contains the cryptographic keys that allows other peers to communicate with it.
 
-In the `![ Network ]`! section of the program, you can monitor announces on the network, initiate conversations with announced peers, and announce your own peer on the network.
+In the `![ Network ]`! section of the program, you can monitor announces on the network, initiate conversations with announced peers, and announce your own peer on the network. You can also connect to nodes on the network and browse information shared by them.
 
 >>Conversations
 
@@ -188,12 +220,111 @@ If no nodes exist on a network, all peers will still be able to communicate dire
 
 '''
 
-TOPIC_CONVERSATIONS = '''Conversations
+TOPIC_HOSTING = '''>Hosting a Node
-=============
+
+To host a node on the network, you must enable it in the configuration file, by setting `*enable_node`* directive to `*yes`*. You should also configure the other node-related parameters such as the node name and announce interval settings. Once node hosting has been enabled in the configuration, Nomad Network will start hosting your node as soon as the program is lauched, and other peers on the network will be able to connect and interact with content on your node.
+
+By default, no content is defined, apart from a short placeholder home page. To learn how to add your own content, read on.
+
+>>Pages
+
+Nomad Network nodes can host pages similar to web pages, that other peers can read and interact with. Pages are written in a compact markup language called `*micron`*. To learn how to write formatted pages with micron, see the `*Markup`* section of this guide (which is, itself, written in micron). Pages can be linked arbitrarily with hyperlinks, that can also link to pages (or other resources) on other nodes.
+
+To add pages to your node, place micron files in the `*pages`* directory of your Nomad Network programs `*storage`* directory. By default, the path to this will be `!~/.nomadnetwork/storage/pages`!. You should probably create the file `!index.mu`! first, as this is the page that will get served by default to a connecting peer.
+
+Pages are static in this version, but the next release of Nomad Network will add the ability to use a preprocessor such as PHP, bash, Python (or whatever you prefer) to generate dynamic pages.
+
+>>Files
+
+Like pages, you can place files you want to make available in the `!~/.nomadnetwork/storage/files`! directory. To let a peer download a file, you should create a link to it in one of your pages.
+
+>>Links and URLs
+
+Links to pages and resources in Nomad Network use a simple URL format. Here is an example:
+
+`!1385edace36466a6b3dd:/page/index.mu`!
+
+The first part is the 10 byte destination address of the node (represented as readable hexadecimal), followed by the `!:`! character. Everything after the `!:`! represents the request path.
+
+By convention, Nomad Network nodes maps all hosted pages under the `!/page`! path, and all hosted files under the `!/file`! path. You can create as many subdirectories for pages and files as you please, and they will be automatically mapped to corresponding request paths.
+
+You can omit the destination address of the node, if you are reffering to a local page or file. You must still keep the `!:`! character. In such a case, the URL to a page could look like this:
+
+`!:/page/other_page.mu`!
+
+The URL to a local file could look like this:
+
+`!:/file/document.pdf`!
+
+Links can be inserted into micron documents. See the `*Markup`* section of this guide for info on how to do so.
+
+'''
+
+TOPIC_CONVERSATIONS = '''>Conversations
 
 Conversations in Nomad Network
 '''
 
+
+TOPIC_FIRST_RUN = '''>First Time Information
+
+Hi there. This first run message will only appear once. It contains a few pointers on getting started with Nomad Network, and getting the most out of the program.
+
+You're currently located in the guide section of the program. I'm sorry I had to drag you here by force, but it will only happen this one time, I promise. If you ever get lost, return here and peruse the list of topics you see on the left. I will do my best to fill it with answers to mostly anything about Nomad Network.
+
+To get the most out of Nomad Network, you will need a terminal that supports UTF-8 and at least 256 colors, ideally true-color. If your terminal supports true-color, you can go to the `![ Config ]`! menu item, launch the editor and change the configuration.
+
+If you don't already have a Nerd Font installed (see https://www.nerdfonts.com/), I also highly recommend to do so, since it will greatly expand the amount of glyphs, icons and graphics that Nomad Network can use. Once you have your terminal set up with a Nerd Font, go to the `![ Config ]`! menu item and enable Nerd Fonts in the configuration instead of normal unicode glyphs.
+
+Nomad Network expects that you are already connected to some form of Reticulum network. That could be as simple as the default UDP-based demo interface on your local ethernet network. This short guide won't go into any details on building, but you will find other entries in the guide that deal with network setup and configuration.
+
+At least, if Nomad Network launches, it means that it is connected to a running Reticulum instance, that should in turn be connected to `*something`*, which should get you started.
+
+For more some more information, you can also read the `*Introduction`* section of this guide.
+
+Now go out there and explore. This is still early days. See what you can find and create.
+
+>>>>>>>>>>>>>>>
+-\u223f
+<
+
+'''
+
+TOPIC_LICENSES = '''>Thanks, Acknowledgements and Licenses
+
+This program uses various other software components, without which Nomad Network would not have been possible. Sincere thanks to the authors and contributors of the following projects
+
+>>>
+ - `!Cryptography.io`! by `*pyca`*
+   https://cryptography.io/
+   BSD License
+
+ - `!Urwid`! by `*Ian Ward`*
+   http://urwid.org/
+   LGPL-2.1 License
+
+ - `!Additional Urwid Widgets`! by `*AFoeee`*
+   https://github.com/AFoeee/additional_urwid_widgets
+   MIT License
+
+ - `!Scrollable`! by `*rndusr`*
+   https://github.com/rndusr/stig/blob/master/stig/tui/scroll.py
+   GPLv3 License
+
+ - `!Configobj`! by `*Michael Foord`*
+   https://github.com/DiffSK/configobj
+   BSD License
+
+ - `!Reticulum Network Stack`! by `*unsignedmark`*
+   https://github.com/markqvist/Reticulum
+   MIT License
+
+ - `!LXMF`! by `*unsignedmark`*
+   https://github.com/markqvist/LXMF
+   MIT License
+'''
+
+
 TOPIC_MARKUP = '''>Outputting Formatted Text
 
 
@@ -218,6 +349,10 @@ With micron you can easily create structured documents and pages with formatting
 
 While micron can output formatted text to even the most basic terminal, there's a few capabilities your terminal `*must`* support to display micron output correctly, and some that, while not strictly necessary, make the experience a lot better.
 
+Formatting such as `_underline`_, `!bold`! or `*italics`* will be displayed if your terminal supports it.
+
+If you are having trouble getting micron output to display correctly, try using `*gnome-terminal`*, which should work with all formatting options out of the box. Most other terminals will work fine as well, but you might have to change some settings to get certain formatting to display correctly.
+
 >>>Encoding
 
 All micron sources are intepreted as UTF-8, and micron assumes it can output UTF-8 characters to the terminal. If your terminal does not support UTF-8, output will be faulty.
@@ -228,7 +363,7 @@ Shading and coloring text and backgrounds is integral to micron output, and whil
 
 >>>Terminal Font
 
-While any font any unicode capable font can be used with micron, it's highly recommended to use a `*"Nerd Font"`* (see https://www.nerdfonts.com/), which will add a lot of extra glyphs and icons to your output.
+While any unicode capable font can be used with micron, it's highly recommended to use a `*"Nerd Font"`* (see https://www.nerdfonts.com/), which will add a lot of extra glyphs and icons to your output.
 
 > A Few Demo Outputs
 
@@ -388,6 +523,36 @@ You can use `B5d5`F222 color `f`B333 `Ff00f`Ff80o`Ffd0r`F9f0m`F0f2a`F0fdt`F07ft`
 ``
 
 
+>Links
+
+Links to pages, files or other resources can be created with the \\`[ tag, which should always be terminated with a closing ]. You can create links with and without labels, it is up to you to control the formatting of links with other tags. Although not strictly necessary, it is good practice to at least format links with underlining.
+
+Here's a few examples:
+
+`Faaa
+`=
+Here is a link without any label: `[1385edace36466a6b3dd:/page/index.mu]
+
+This is a `[labeled link`1385edace36466a6b3dd:/page/index.mu] to the same page, but it's hard to see if you don't know it
+
+Here is `F00a`_`[a more visible link`1385edace36466a6b3dd:/page/index.mu]`_`f
+`=
+``
+
+The above markup produces the following output:
+
+`Faaa`B333
+
+Here is a link without any label: `[1385edace36466a6b3dd:/page/index.mu]
+
+This is a `[labeled link`1385edace36466a6b3dd:/page/index.mu] to the same page, but it's hard to see if you don't know it
+
+Here is `F00f`_`[a more visible link`1385edace36466a6b3dd:/page/index.mu]`_`f
+
+``
+
+When links like these are displayed in the built-in browser, clicking on them or activating them using the keyboard will cause the browser to load the specified URL.
+
 >Literals
 
 To display literal content, for example source-code, or blocks of text that should not be interpreted by micron, you can use literal blocks, specified by the \\`= tag. Below is the source code of this entire document, presented as a literal block.
@@ -399,53 +564,11 @@ To display literal content, for example source-code, or blocks of text that shou
 TOPIC_MARKUP += TOPIC_MARKUP.replace("`=", "\\`=") + "[ micron source for document goes here, we don't want infinite recursion now, do we? ]\n\\`="
 TOPIC_MARKUP += "\n`=\n\n>Closing Remarks\n\nIf you made it all the way here, you should be well equipped to write documents and pages using micron. Thank you for staying with me.\n\n`c\U0001F332\n"
 
-TOPIC_FIRST_RUN = '''>First Time Information
-
-Hi there. This first run message will only appear once. It contains a few pointers on getting started with Nomad Network, and getting the most out of the program. You're currently located in the guide section of the program. I'm sorry I had to drag you here by force, but it will only happen this one time, I promise. If you ever get lost, return here and peruse the list of topics you see on the left. I will do my best to fill it with answers to mostly anything about Nomad Network.
-
-To get the most out of Nomad Network, you will need a terminal that supports UTF-8 and at least 256 colors, ideally true-color. By default, Nomad Network starts in low-color mode. It does this for the sake of compatibility, but it does look rather ugly. If your terminal supports true-color or just 256 colors, you should go to the `![ Config ]`! menu item, launch the editor and change the configuration to use a high-color mode.
-
-If you don't already have a Nerd Font installed (see https://www.nerdfonts.com/), I also highly recommend to do so, since it will greatly expand the amount of glyphs, icons and graphics that Nomad Network can use.
-
-Nomad Network expects that you are already connected to some form of Reticulum network. That could be as simple as the default UDP-based demo interface on your local ethernet network, or as advanced as some elaborate hybrid RF and free-space optical laser network. This short guide won't go into any details on that, but you will find other entries in the guide that deal with network setup and configuration.
-
-At least, if Nomad Network launches, it means that it is connected to a running Reticulum instance, that should in turn be connected to `*something`*, which should get you started.
-
-Now go out there and explore. This is still early days. See what you can find and create.
-
->>>>>>>>>>>>>>>
--\u223f
-<
-
-'''
-
-TOPIC_LICENSES = '''>Thanks, Acknowledgements and Licenses
-
-This program uses various other software components, without which Nomad Network would not have been possible. Sincere thanks to the authors and contributors of the following projects
-
->>>
- - `!Cryptography.io`! by `*pyca`*
-   BSD License
-
- - `!Additional Urwid Widgets`! by `*AFoeee`*
-   MIT License
-
- - `!Scrollable`! by `*rndusr`*
-   GPLv3 License
-
- - `!Configobj`! by `*Michael Foord`*
-   BSD License
-
- - `!Reticulum Network Stack`! by `*unsignedmark`*
-   MIT License
-
- - `!LXMF`! by `*unsignedmark`*
-   MIT License
-'''
 
 TOPICS = {
     "Introduction": TOPIC_INTRODUCTION,
     "Conversations": TOPIC_CONVERSATIONS,
+    "Hosting a Node": TOPIC_HOSTING,
     "Markup": TOPIC_MARKUP,
     "First Run": TOPIC_FIRST_RUN,
     "Credits & Licenses": TOPIC_LICENSES,

nomadnet/ui/textui/Main.py

@@ -21,7 +21,10 @@ class SubDisplays():
         self.log_display = LogDisplay(self.app)
         self.guide_display = GuideDisplay(self.app)
 
-        self.active_display = self.conversations_display
+        if app.firstrun:
+            self.active_display = self.guide_display
+        else:
+            self.active_display = self.conversations_display
 
     def active(self):
         return self.active_display

nomadnet/ui/textui/MicronParser.py

@@ -1,27 +1,34 @@
 import nomadnet
 import urwid
+import time
+from urwid.util import is_mouse_press
+from urwid.text_layout import calc_coords
 import re
 
+DEFAULT_FG = "ddd"
+DEFAULT_BG = "default"
+
 STYLES = {
-    "plain":    { "fg": "bbb", "bg": "default", "bold": False, "underline": False, "italic": False },
+    "plain":    { "fg": DEFAULT_FG, "bg": DEFAULT_BG, "bold": False, "underline": False, "italic": False },
     "heading1": { "fg": "222", "bg": "bbb", "bold": False, "underline": False, "italic": False },
     "heading2": { "fg": "111", "bg": "999", "bold": False, "underline": False, "italic": False },
     "heading3": { "fg": "000", "bg": "777", "bold": False, "underline": False, "italic": False },
 }
 
 SYNTH_STYLES = []
+SYNTH_SPECS  = {}
 
 SECTION_INDENT = 2
 INDENT_RIGHT   = 1
 
-def markup_to_attrmaps(markup):
+def markup_to_attrmaps(markup, url_delegate = None):
     attrmaps = []
 
     state = {
         "literal": False,
         "depth": 0,
-        "fg_color": "default",
+        "fg_color": DEFAULT_FG,
-        "bg_color": "default",
+        "bg_color": DEFAULT_BG,
         "formatting": {
             "bold": False,
             "underline": False,
@@ -39,7 +46,7 @@ def markup_to_attrmaps(markup):
 
     for line in lines:
         if len(line) > 0:
-            display_widget = parse_line(line, state)
+            display_widget = parse_line(line, state, url_delegate)
         else:
             display_widget = urwid.Text("")
         
@@ -50,7 +57,7 @@ def markup_to_attrmaps(markup):
     return attrmaps
 
 
-def parse_line(line, state):
+def parse_line(line, state, url_delegate):
     if len(line) > 0:
         first_char = line[0]
 
@@ -68,7 +75,7 @@ def parse_line(line, state):
             # Check for section heading reset
             elif first_char == "<":
                 state["depth"] = 0
-                return parse_line(line[1:], state)
+                return parse_line(line[1:], state, url_delegate)
 
             # Check for section headings
             elif first_char == ">":
@@ -88,7 +95,7 @@ def parse_line(line, state):
                     style_to_state(style, state)
 
                     heading_style = make_style(state)
-                    output = make_output(state, line)
+                    output = make_output(state, line, url_delegate)
                     
                     style_to_state(latched_style, state)
 
@@ -114,13 +121,18 @@ def parse_line(line, state):
                 else:
                     return urwid.Padding(urwid.Divider(divider_char), left=left_indent(state), right=right_indent(state))
 
-        output = make_output(state, line)
+        output = make_output(state, line, url_delegate)
 
         if output != None:
-            if state["depth"] == 0:
+            if url_delegate != None:
-                return urwid.Text(output, align=state["align"])
+                text_widget = LinkableText(output, align=state["align"], delegate=url_delegate)
             else:
-                return urwid.Padding(urwid.Text(output, align=state["align"]), left=left_indent(state), right=right_indent(state))
+                text_widget = urwid.Text(output, align=state["align"])
+
+            if state["depth"] == 0:
+                return text_widget
+            else:
+                return urwid.Padding(text_widget, left=left_indent(state), right=right_indent(state))
         else:
             return None
     else:
@@ -154,7 +166,7 @@ def make_style(state):
     def mono_color(fg, bg):
         return "default"
     def low_color(color):
-        # TODO: Implement
+        # TODO: Implement low-color mapper
         return "default"
     def high_color(color):
         if color == "default":
@@ -180,11 +192,15 @@ def make_style(state):
     if not name in SYNTH_STYLES:
         screen = nomadnet.NomadNetworkApp.get_shared_instance().ui.screen
         screen.register_palette_entry(name, low_color(fg)+format_string,low_color(bg),mono_color(fg, bg)+format_string,high_color(fg)+format_string,high_color(bg))
+        
+        synth_spec = screen._palette[name]
         SYNTH_STYLES.append(name)
+        if not name in SYNTH_SPECS:
+            SYNTH_SPECS[name] = synth_spec
 
     return name
 
-def make_output(state, line):
+def make_output(state, line, url_delegate):
     output = []
     if state["literal"]:
         if line == "\\`=":
@@ -213,20 +229,20 @@ def make_output(state, line):
                             state["fg_color"] = color
                             skip = 3
                     elif c == "f":
-                        state["fg_color"] = "default"
+                        state["fg_color"] = DEFAULT_FG
                     elif c == "B":
                         if len(line) >= i+4:
                             color = line[i+1:i+4]
                             state["bg_color"] = color
                             skip = 3
                     elif c == "b":
-                        state["bg_color"] = "default"
+                        state["bg_color"] = DEFAULT_BG
                     elif c == "`":
                         state["formatting"]["bold"]      = False 
                         state["formatting"]["underline"] = False
                         state["formatting"]["italic"]    = False
-                        state["fg_color"] = "default"
+                        state["fg_color"] = DEFAULT_FG
-                        state["bg_color"] = "default"
+                        state["bg_color"] = DEFAULT_BG
                         state["align"] = state["default_align"]
                     elif c == "c":
                         if state["align"] != "center":
@@ -246,6 +262,57 @@ def make_output(state, line):
                     elif c == "a":
                         state["align"] = state["default_align"]
 
+                    elif c == "[":
+                        endpos = line[i:].find("]")
+                        if endpos == -1:
+                            pass
+                        else:
+                            link_data = line[i+1:i+endpos]
+                            skip = endpos
+
+                            link_components = link_data.split("`")
+                            if len(link_components) == 1:
+                                link_label = ""
+                                link_url = link_data
+                            elif len(link_components) == 2:
+                                link_label = link_components[0]
+                                link_url = link_components[1]
+                            else:
+                                link_url = ""
+                                link_label = ""
+
+                            if len(link_url) != 0:
+                                if link_label == "":
+                                    link_label = link_url
+
+                                # First generate output until now
+                                if len(part) > 0:
+                                    output.append(make_part(state, part))
+
+                                cm = nomadnet.NomadNetworkApp.get_shared_instance().ui.colormode
+
+                                specname = make_style(state)
+                                speclist = SYNTH_SPECS[specname]
+
+                                if cm == 1:
+                                    orig_spec = speclist[0]
+                                elif cm == 16:
+                                    orig_spec = speclist[1]
+                                elif cm == 88:
+                                    orig_spec = speclist[2]
+                                elif cm == 256:
+                                    orig_spec = speclist[3]
+                                elif cm == 2**24:
+                                    orig_spec = speclist[4]
+
+                                if url_delegate != None:
+                                    linkspec = LinkSpec(link_url, orig_spec)
+                                    output.append((linkspec, link_label))
+                                else:
+                                    output.append(make_part(state, link_label)) 
+
+                                
+
                     mode = "text"
                     if len(part) > 0:
                         output.append(make_part(state, part))
@@ -272,4 +339,155 @@ def make_output(state, line):
     if len(output) > 0:
         return output
     else:
-        return None
+        return None
+
+
+class LinkSpec(urwid.AttrSpec):
+    def __init__(self, link_target, orig_spec):
+        self.link_target = link_target
+
+        urwid.AttrSpec.__init__(self, orig_spec.foreground, orig_spec.background)
+
+
+class LinkableText(urwid.Text):
+    ignore_focus = False
+    _selectable = True
+
+    signals = ["click", "change"]
+
+    def __init__(self, text, align=None, cursor_position=0, delegate=None):
+        self.__super.__init__(text, align=align)
+        self.delegate = delegate
+        self._cursor_position = 0
+        self.key_timeout = 3
+        if self.delegate != None:
+            self.delegate.last_keypress = 0
+
+    def handle_link(self, link_target):
+        if self.delegate != None:
+            self.delegate.handle_link(link_target)
+
+
+    def find_next_part_pos(self, pos, part_positions):
+        for position in part_positions:
+            if position > pos:
+                return position
+        return pos
+
+    def find_prev_part_pos(self, pos, part_positions):
+        nextpos = pos
+        for position in part_positions:
+            if position < pos:
+                nextpos = position
+        return nextpos
+
+    def find_item_at_pos(self, pos):
+        total = 0
+        text, parts = self.get_text()
+        for i, info in enumerate(parts):
+            style, length = info
+            if total <= pos < length+total:
+                return style
+
+            total += length
+
+        return None
+
+    def peek_link(self):
+        item = self.find_item_at_pos(self._cursor_position)
+        if item != None:
+            if isinstance(item, LinkSpec):
+                if self.delegate != None:
+                    self.delegate.marked_link(item.link_target)
+            else:
+                if self.delegate != None:
+                    self.delegate.marked_link(None)
+
+
+    def keypress(self, size, key):
+        part_positions = [0]
+        parts = []
+        total = 0
+        text, parts = self.get_text()
+        for i, info in enumerate(parts):
+            style_name, length = info
+            part_positions.append(length+total)
+            total += length
+
+
+        if self.delegate != None:
+            self.delegate.last_keypress = time.time()
+            self._invalidate()
+            nomadnet.NomadNetworkApp.get_shared_instance().ui.loop.set_alarm_in(self.key_timeout, self.kt_event)
+
+        if self._command_map[key] == urwid.ACTIVATE:
+            item = self.find_item_at_pos(self._cursor_position)
+            if item != None:
+                if isinstance(item, LinkSpec):
+                    self.handle_link(item.link_target)
+
+        elif key == "up":
+            self._cursor_position = 0
+            return key
+        
+        elif key == "down":
+            self._cursor_position = 0
+            return key
+        
+        elif key == "right":
+            self._cursor_position = self.find_next_part_pos(self._cursor_position, part_positions)
+            self._invalidate()
+        
+        elif key == "left":
+            if self._cursor_position > 0:
+                self._cursor_position = self.find_prev_part_pos(self._cursor_position, part_positions)
+                self._invalidate()
+
+            else:
+                if self.delegate != None:
+                    self.delegate.micron_released_focus()
+
+        else:
+            return key
+
+    def kt_event(self, loop, user_data):
+        self._invalidate()
+
+    def render(self, size, focus=False):
+        now = time.time()
+        c = self.__super.render(size, focus)
+
+        if focus and (self.delegate == None or now < self.delegate.last_keypress+self.key_timeout):
+            c = urwid.CompositeCanvas(c)
+            c.cursor = self.get_cursor_coords(size)
+            if self.delegate != None:
+                self.peek_link()
+
+        return c
+
+    def get_cursor_coords(self, size):
+            if self._cursor_position > len(self.text):
+                return None
+
+            (maxcol,) = size
+            trans = self.get_line_translation(maxcol)
+            x, y = calc_coords(self.text, trans, self._cursor_position)
+            if maxcol <= x:
+                return None
+            return x, y
+
+    def mouse_event(self, size, event, button, x, y, focus):
+        if button != 1 or not is_mouse_press(event):
+            return False
+        else:
+            pos = (y * size[0]) + x
+            self._cursor_position = pos
+            item = self.find_item_at_pos(self._cursor_position)
+            if item != None:
+                if isinstance(item, LinkSpec):
+                    self.handle_link(item.link_target)
+
+            self._invalidate()
+            self._emit("change")
+            
+            return True

nomadnet/ui/textui/Network.py

@@ -1,16 +1,20 @@
 import RNS
 import urwid
 import nomadnet
+import time
 from datetime import datetime
 from nomadnet.Directory import DirectoryEntry
 from nomadnet.vendor.additional_urwid_widgets import IndicativeListBox, MODIFIER_KEY
 
+from .Browser import Browser
+
 class NetworkDisplayShortcuts():
     def __init__(self, app):
         self.app = app
         g = app.ui.glyphs
 
-        self.widget = urwid.AttrMap(urwid.Text("[C-"+g["arrow_u"]+g["arrow_d"]+"] Navigate announces"), "shortcutbar")
+        self.widget = urwid.AttrMap(urwid.Text("[C-l] Toggle Nodes/Announces view  [C-x] Remove entry  [C-w] Disconnect remote"), "shortcutbar")
+        #   "[C-"+g["arrow_u"]+g["arrow_d"]+"] Navigate Lists"
 
 
 class DialogLineBox(urwid.LineBox):
@@ -60,6 +64,13 @@ class AnnounceInfo(urwid.WidgetWrap):
         trust_str    = ""
         display_str  = self.app.directory.simplest_display_str(source_hash)
         addr_str     = "<"+RNS.hexrep(source_hash, delimit=False)+">"
+        is_node      = announce[3]
+
+        if is_node:
+            type_string = g["node"] + " Node"
+        else:
+            type_string = g["peer"] + " Peer"
+
         try:
             data_str = announce[2].decode("utf-8")
             data_style = ""
@@ -93,7 +104,11 @@ class AnnounceInfo(urwid.WidgetWrap):
 
         def show_announce_stream(sender):
             options = self.parent.left_pile.options(height_type="weight", height_amount=1)
-            self.parent.left_pile.contents[1] = (AnnounceStream(self.app, self.parent), options)
+            self.parent.left_pile.contents[0] = (self.parent.announce_stream_display, options)
+
+        def connect(sender):
+            self.parent.browser.retrieve_url(RNS.hexrep(source_hash, delimit=False))
+            show_announce_stream(None)
 
         def converse(sender):
             show_announce_stream(None)
@@ -116,15 +131,31 @@ class AnnounceInfo(urwid.WidgetWrap):
             except Exception as e:
                 RNS.log("Error while starting conversation from announce. The contained exception was: "+str(e), RNS.LOG_ERROR)
 
-        pile = urwid.Pile([
+        if is_node:
+            type_button = ("weight", 0.45, urwid.Button("Connect", on_press=connect))
+        else:
+            type_button = ("weight", 0.45, urwid.Button("Converse", on_press=converse))
+
+        pile_widgets = [
+            urwid.Text("Time  : "+ts_string, align="left"),
             urwid.Text("Addr  : "+addr_str, align="left"),
+            urwid.Text("Type  : "+type_string, align="left"),
             urwid.Text("Name  : "+display_str, align="left"),
             urwid.Text(["Trust : ", (style, trust_str)], align="left"),
             urwid.Divider(g["divider1"]),
             urwid.Text(["Announce Data: \n", (data_style, data_str)], align="left"),
             urwid.Divider(g["divider1"]),
-            urwid.Columns([("weight", 0.45, urwid.Button("Back", on_press=show_announce_stream)), ("weight", 0.1, urwid.Text("")), ("weight", 0.45, urwid.Button("Converse", on_press=converse))])
+            urwid.Columns([("weight", 0.45, urwid.Button("Back", on_press=show_announce_stream)), ("weight", 0.1, urwid.Text("")), type_button])
-        ])
+        ]
+
+        if is_node:
+            node_ident = RNS.Identity.recall(source_hash)
+            op_hash = RNS.Destination.hash_from_name_and_identity("lxmf.delivery", node_ident)
+            op_str = self.app.directory.simplest_display_str(op_hash)
+            operator_entry = urwid.Text("Oprtr : "+op_str, align="left")
+            pile_widgets.insert(4, operator_entry)
+
+        pile = urwid.Pile(pile_widgets)
 
         self.display_widget = urwid.Filler(pile, valign="top", height="pack")
 
@@ -133,15 +164,26 @@ class AnnounceInfo(urwid.WidgetWrap):
 
 class AnnounceStreamEntry(urwid.WidgetWrap):
     def __init__(self, app, announce):
+        full_time_format = "%Y-%m-%d %H:%M:%S"
+        date_time_format = "%Y-%m-%d"
+        time_time_format = "%H:%M:%S"
+        short_time_format = "%Y-%m-%d %H:%M"
+
         timestamp = announce[0]
         source_hash = announce[1]
+        is_node = announce[3]
         self.app = app
         self.timestamp = timestamp
         time_format = app.time_format
         dt = datetime.fromtimestamp(self.timestamp)
-        ts_string = dt.strftime(time_format)
+        dtn = datetime.fromtimestamp(time.time())
         g = self.app.ui.glyphs
 
+        if dt.strftime(date_time_format) == dtn.strftime(date_time_format):
+            ts_string = dt.strftime(time_time_format)
+        else:
+            ts_string = dt.strftime(short_time_format)
+
         trust_level  = self.app.directory.trust_level(source_hash)
         display_str = self.app.directory.simplest_display_str(source_hash)
 
@@ -166,7 +208,12 @@ class AnnounceStreamEntry(urwid.WidgetWrap):
             style         = "list_untrusted"
             focus_style   = "list_focus_untrusted"
 
-        widget = ListEntry(ts_string+" "+display_str)
+        if is_node:
+            type_symbol = g["node"]
+        else:
+            type_symbol = g["peer"]
+
+        widget = ListEntry(ts_string+" "+type_symbol+" "+display_str)
         urwid.connect_signal(widget, "click", self.display_announce, announce)
 
         self.display_widget = urwid.AttrMap(widget, style, focus_style)
@@ -176,7 +223,10 @@ class AnnounceStreamEntry(urwid.WidgetWrap):
         parent = self.app.ui.main_display.sub_displays.network_display
         info_widget = AnnounceInfo(announce, parent, self.app)
         options = parent.left_pile.options(height_type="weight", height_amount=1)
-        parent.left_pile.contents[1] = (info_widget, options)
+        parent.left_pile.contents[0] = (info_widget, options)
+
+    def timestamp(self):
+        return self.timestamp
 
 class AnnounceStream(urwid.WidgetWrap):
     def __init__(self, app, parent):
@@ -185,25 +235,38 @@ class AnnounceStream(urwid.WidgetWrap):
         self.started = False
         self.timeout = self.app.config["textui"]["animation_interval"]*2
         self.ilb = None
+        self.no_content = True
         
         self.added_entries = []
         self.widget_list = []
         self.update_widget_list()
 
-        wlt = [AnnounceStreamEntry(self.app, e) for e in self.app.directory.announce_stream]
         self.ilb = IndicativeListBox(
             self.widget_list,
-            #wlt,
             on_selection_change=self.list_selection,
             initialization_is_selection_change=False,
-            modifier_key=MODIFIER_KEY.CTRL,
+            #modifier_key=MODIFIER_KEY.CTRL,
             #highlight_offFocus="list_off_focus"
         )
 
         self.display_widget = self.ilb
         urwid.WidgetWrap.__init__(self, urwid.LineBox(self.display_widget, title="Announce Stream"))
 
+    def keypress(self, size, key):
+        if key == "up" and (self.no_content or self.ilb.first_item_is_selected()):
+            nomadnet.NomadNetworkApp.get_shared_instance().ui.main_display.frame.set_focus("header")
+        elif key == "ctrl x":
+            self.delete_selected_entry()
+            
+        return super(AnnounceStream, self).keypress(size, key)
+
+    def delete_selected_entry(self):
+        if self.ilb.get_selected_item() != None:
+            self.app.directory.remove_announce_with_timestamp(self.ilb.get_selected_item().original_widget.timestamp)
+            self.rebuild_widget_list()
+
     def rebuild_widget_list(self):
+        self.no_content = True
         self.added_entries = []
         self.widget_list = []
         self.update_widget_list()
@@ -215,13 +278,22 @@ class AnnounceStream(urwid.WidgetWrap):
                 self.added_entries.insert(0, e[0])
                 new_entries.insert(0, e)
 
-        new_widgets = [AnnounceStreamEntry(self.app, e) for e in new_entries]
+        for e in new_entries:
-        for nw in new_widgets:
+            nw = AnnounceStreamEntry(self.app, e)
+            nw.timestamp = e[0]
             self.widget_list.insert(0, nw)
 
-        if len(new_widgets) > 0:
+        if len(new_entries) > 0:
+            self.no_content = False
             if self.ilb != None:
                 self.ilb.set_body(self.widget_list)
+        else:
+            if len(self.widget_list) == 0:
+                self.no_content = True
+            
+            if self.ilb != None:
+                self.ilb.set_body(self.widget_list)
+
 
     def list_selection(self, arg1, arg2):
         pass
@@ -267,14 +339,23 @@ class SelectText(urwid.Text):
         self._emit('click')
         return True
 
+class ListDialogLineBox(urwid.LineBox):
+    def keypress(self, size, key):
+        if key == "esc":
+            self.delegate.close_list_dialogs()
+        else:
+            return super(ListDialogLineBox, self).keypress(size, key)
+
 class KnownNodes(urwid.WidgetWrap):
     def __init__(self, app):
         self.app = app
         self.node_list = app.directory.known_nodes()
         g = self.app.ui.glyphs
+
+        self.widget_list = self.make_node_widgets()
         
         self.ilb = IndicativeListBox(
-            self.make_node_widgets(),
+            self.widget_list,
             on_selection_change=self.node_list_selection,
             initialization_is_selection_change=False,
             highlight_offFocus="list_off_focus"
@@ -287,13 +368,16 @@ class KnownNodes(urwid.WidgetWrap):
         else:
             self.no_content = True
             widget_style = "inactive_text"
-            self.display_widget = urwid.Pile([urwid.Text(("warning_text", g["info"]+"\n"), align="center"), SelectText(("warning_text", "Currently, no nodes are known\n\n"), align="center")])
+            self.pile = urwid.Pile([urwid.Text(("warning_text", g["info"]+"\n"), align="center"), SelectText(("warning_text", "Currently, no nodes are known\n\n"), align="center")])
+            self.display_widget = urwid.Filler(self.pile, valign="top", height="pack")
 
         urwid.WidgetWrap.__init__(self, urwid.AttrMap(urwid.LineBox(self.display_widget, title="Known Nodes"), widget_style))
 
     def keypress(self, size, key):
-        if key == "up" and (self.no_content or self.ilb.top_is_visible):
+        if key == "up" and (self.no_content or self.ilb.first_item_is_selected()):
             nomadnet.NomadNetworkApp.get_shared_instance().ui.main_display.frame.set_focus("header")
+        elif key == "ctrl x":
+            self.delete_selected_entry()
             
         return super(KnownNodes, self).keypress(size, key)
 
@@ -301,16 +385,126 @@ class KnownNodes(urwid.WidgetWrap):
     def node_list_selection(self, arg1, arg2):
         pass
 
+    def connect_node(self, event, node):
+        source_hash = node.source_hash
+        trust_level = node.trust_level
+        trust_level  = self.app.directory.trust_level(source_hash)
+        display_str = self.app.directory.simplest_display_str(source_hash)
+
+        def dismiss_dialog(sender):
+            self.delegate.close_list_dialogs()
+
+        def confirmed(sender):
+            self.delegate.browser.retrieve_url(RNS.hexrep(source_hash, delimit=False))
+            self.delegate.close_list_dialogs()
+
+
+        dialog = ListDialogLineBox(
+            urwid.Pile([
+                urwid.Text("Connect to node\n"+self.app.directory.simplest_display_str(source_hash)+"\n", align="center"),
+                urwid.Columns([("weight", 0.45, urwid.Button("Yes", on_press=confirmed)), ("weight", 0.1, urwid.Text("")), ("weight", 0.45, urwid.Button("No", on_press=dismiss_dialog))])
+            ]), title="?"
+        )
+        dialog.delegate = self.delegate
+        bottom = self
+
+        overlay = urwid.Overlay(dialog, bottom, align="center", width=("relative", 100), valign="middle", height="pack", left=2, right=2)
+
+        options = self.delegate.left_pile.options("weight", 1)
+        self.delegate.left_pile.contents[0] = (overlay, options)
+
+    def delete_selected_entry(self):
+        source_hash = self.ilb.get_selected_item().original_widget.source_hash
+
+        def dismiss_dialog(sender):
+            self.delegate.close_list_dialogs()
+
+        def confirmed(sender):
+            self.app.directory.forget(source_hash)
+            self.rebuild_widget_list()
+            self.delegate.close_list_dialogs()
+
+
+        dialog = ListDialogLineBox(
+            urwid.Pile([
+                urwid.Text("Delete Node\n"+self.app.directory.simplest_display_str(source_hash)+"\n", align="center"),
+                urwid.Columns([("weight", 0.45, urwid.Button("Yes", on_press=confirmed)), ("weight", 0.1, urwid.Text("")), ("weight", 0.45, urwid.Button("No", on_press=dismiss_dialog))])
+            ]), title="?"
+        )
+        dialog.delegate = self.delegate
+        bottom = self
+
+        overlay = urwid.Overlay(dialog, bottom, align="center", width=("relative", 100), valign="middle", height="pack", left=2, right=2)
+
+        options = self.delegate.left_pile.options("weight", 1)
+        self.delegate.left_pile.contents[0] = (overlay, options)
+
+
+    def rebuild_widget_list(self):
+        self.node_list = self.app.directory.known_nodes()
+        self.widget_list = self.make_node_widgets()
+        self.ilb.set_body(self.widget_list)
+        if len(self.widget_list) > 0:
+            self.no_content = False
+        else:
+            self.no_content = True
+            self.delegate.reinit_known_nodes()
+
     def make_node_widgets(self):
         widget_list = []
         for node_entry in self.node_list:
             # TODO: Implement this
-            widget_list.append(ListEntry("Node "+RNS.prettyhexrep(node_entry.source_hash)))
+            ne = NodeEntry(self.app, node_entry, self)
+            ne.source_hash = node_entry.source_hash
+            widget_list.append(ne)
 
         # TODO: Sort list
         return widget_list
 
 
+
+class NodeEntry(urwid.WidgetWrap):
+    def __init__(self, app, node, delegate):
+        source_hash = node.source_hash
+        trust_level = node.trust_level
+
+        self.app = app
+        g = self.app.ui.glyphs
+
+        trust_level  = self.app.directory.trust_level(source_hash)
+        display_str = self.app.directory.simplest_display_str(source_hash)
+
+        if trust_level == DirectoryEntry.UNTRUSTED:
+            symbol        = g["cross"]
+            style         = "list_untrusted"
+            focus_style   = "list_focus_untrusted"
+        elif trust_level == DirectoryEntry.UNKNOWN:
+            symbol        = g["unknown"]
+            style         = "list_unknown"
+            focus_style   = "list_focus"
+        elif trust_level == DirectoryEntry.TRUSTED:
+            symbol        = g["check"]
+            style         = "list_trusted"
+            focus_style   = "list_focus_trusted"
+        elif trust_level == DirectoryEntry.WARNING:
+            symbol        = g["warning"]
+            style         = "list_warning"
+            focus_style   = "list_focus"
+        else:
+            symbol        = g["warning"]
+            style         = "list_untrusted"
+            focus_style   = "list_focus_untrusted"
+
+        type_symbol = g["node"]
+        
+        widget = ListEntry(type_symbol+" "+display_str)
+        urwid.connect_signal(widget, "click", delegate.connect_node, node)
+
+        self.display_widget = urwid.AttrMap(widget, style, focus_style)
+        self.display_widget.source_hash = source_hash
+        urwid.WidgetWrap.__init__(self, self.display_widget)
+
+
 class AnnounceTime(urwid.WidgetWrap):
     def __init__(self, app):
         self.started = False
@@ -361,7 +555,7 @@ class LocalPeer(urwid.WidgetWrap):
         def save_query(sender):
             def dismiss_dialog(sender):
                 self.dialog_open = False
-                self.parent.left_pile.contents[3] = (LocalPeer(self.app, self.parent), options)
+                self.parent.left_pile.contents[2] = (LocalPeer(self.app, self.parent), options)
 
             self.app.set_display_name(e_name.get_edit_text())
 
@@ -378,13 +572,13 @@ class LocalPeer(urwid.WidgetWrap):
             overlay = dialog
             options = self.parent.left_pile.options(height_type="pack", height_amount=None)
             self.dialog_open = True
-            self.parent.left_pile.contents[3] = (overlay, options)
+            self.parent.left_pile.contents[2] = (overlay, options)
 
         def announce_query(sender):
             def dismiss_dialog(sender):
                 self.dialog_open = False
                 options = self.parent.left_pile.options(height_type="pack", height_amount=None)
-                self.parent.left_pile.contents[3] = (LocalPeer(self.app, self.parent), options)
+                self.parent.left_pile.contents[2] = (LocalPeer(self.app, self.parent), options)
 
             self.app.announce_now()
 
@@ -402,11 +596,11 @@ class LocalPeer(urwid.WidgetWrap):
             
             self.dialog_open = True
             options = self.parent.left_pile.options(height_type="pack", height_amount=None)
-            self.parent.left_pile.contents[3] = (overlay, options)
+            self.parent.left_pile.contents[2] = (overlay, options)
 
         def node_settings_query(sender):
             options = self.parent.left_pile.options(height_type="pack", height_amount=None)
-            self.parent.left_pile.contents[3] = (self.parent.node_settings_display, options)
+            self.parent.left_pile.contents[2] = (self.parent.node_settings_display, options)
 
         if LocalPeer.announce_timer == None:
             self.t_last_announce = AnnounceTime(self.app)
@@ -443,7 +637,7 @@ class NodeSettings(urwid.WidgetWrap):
 
         def show_peer_info(sender):
             options = self.parent.left_pile.options(height_type="pack", height_amount=None)
-            self.parent.left_pile.contents[3] = (LocalPeer(self.app, self.parent), options)
+            self.parent.left_pile.contents[2] = (LocalPeer(self.app, self.parent), options)
 
         widget_style = "inactive_text"
         pile = urwid.Pile([
@@ -518,6 +712,18 @@ class NetworkStats(urwid.WidgetWrap):
         self.w_heard_peers.start()
         self.w_known_nodes.start()
 
+
+
+class NetworkLeftPile(urwid.Pile):
+    def keypress(self, size, key):
+        if key == "ctrl l":
+            self.parent.toggle_list()
+        elif key == "ctrl w":
+            self.parent.browser.disconnect()
+        else:
+            return super(NetworkLeftPile, self).keypress(size, key)
+
+
 class NetworkDisplay():
     list_width = 0.33
 
@@ -525,21 +731,27 @@ class NetworkDisplay():
         self.app = app
         g = self.app.ui.glyphs
 
+        self.browser = Browser(self.app, "nomadnetwork", "node", auth_identity = self.app.identity, delegate = self)
+
         self.known_nodes_display = KnownNodes(self.app)
         self.network_stats_display = NetworkStats(self.app, self)
         self.announce_stream_display = AnnounceStream(self.app, self)
         self.local_peer_display = LocalPeer(self.app, self)
         self.node_settings_display = NodeSettings(self.app, self)
 
-        self.left_pile = urwid.Pile([
+        self.known_nodes_display.delegate = self
-            ("pack", self.known_nodes_display),
+
-            ("weight", 1, self.announce_stream_display),
+        self.list_display = 1
+        self.left_pile = NetworkLeftPile([
+            ("weight", 1, self.known_nodes_display),
             ("pack", self.network_stats_display),
             ("pack", self.local_peer_display),
         ])
 
+        self.left_pile.parent = self
+
         self.left_area = self.left_pile
-        self.right_area = urwid.AttrMap(urwid.LineBox(urwid.Filler(urwid.Text("Disconnected\n"+g["arrow_l"]+"  "+g["arrow_r"], align="center"), "middle"), title="Remote Node"), "inactive_text")
+        self.right_area = self.browser.display_widget
 
         self.columns = urwid.Columns(
             [
@@ -552,6 +764,33 @@ class NetworkDisplay():
         self.shortcuts_display = NetworkDisplayShortcuts(self.app)
         self.widget = self.columns
 
+    def toggle_list(self):
+        if self.list_display != 0:
+            options = self.left_pile.options(height_type="weight", height_amount=1)
+            self.left_pile.contents[0] = (self.announce_stream_display, options)
+            self.list_display = 0
+        else:
+            options = self.left_pile.options(height_type="weight", height_amount=1)
+            self.left_pile.contents[0] = (self.known_nodes_display, options)
+            self.list_display = 1
+
+    def focus_lists(self):
+        self.columns.focus_position = 0
+
+    def reinit_known_nodes(self):
+        self.known_nodes_display = KnownNodes(self.app)
+        self.known_nodes_display.delegate = self
+        self.close_list_dialogs()
+        self.announce_stream_display.rebuild_widget_list()
+
+    def close_list_dialogs(self):
+        if self.list_display == 0:
+            options = self.left_pile.options(height_type="weight", height_amount=1)
+            self.left_pile.contents[0] = (self.announce_stream_display, options)
+        else:
+            options = self.left_pile.options(height_type="weight", height_amount=1)
+            self.left_pile.contents[0] = (self.known_nodes_display, options)
+
     def start(self):
         self.local_peer_display.start()
         self.network_stats_display.start()
@@ -562,6 +801,10 @@ class NetworkDisplay():
 
     def directory_change_callback(self):
         self.announce_stream_display.rebuild_widget_list()
+        if self.known_nodes_display.no_content:
+            self.reinit_known_nodes()
+        else:
+            self.known_nodes_display.rebuild_widget_list()
 
 
 

nomadnet/vendor/Scrollable.py

@@ -31,7 +31,7 @@ class Scrollable(urwid.WidgetDecoration):
     def selectable(self):
         return True
 
-    def __init__(self, widget):
+    def __init__(self, widget, force_forward_keypress = False):
         """Box widget that makes a fixed or flow widget vertically scrollable
 
         TODO: Focusable widgets are handled, including switching focus, but
@@ -49,6 +49,7 @@ class Scrollable(urwid.WidgetDecoration):
         self._forward_keypress = None
         self._old_cursor_coords = None
         self._rows_max_cached = 0
+        self.force_forward_keypress = force_forward_keypress
         self.__super.__init__(widget)
 
     def render(self, size, focus=False):
@@ -127,7 +128,7 @@ class Scrollable(urwid.WidgetDecoration):
 
     def keypress(self, size, key):
         # Maybe offer key to original widget
-        if self._forward_keypress:
+        if self._forward_keypress or self.force_forward_keypress:
             ow = self._original_widget
             ow_size = self._get_original_widget_size(size)
 
@@ -211,7 +212,7 @@ class Scrollable(urwid.WidgetDecoration):
         # If the cursor was moved by the most recent keypress, adjust trim_top
         # so that the new cursor position is within the displayed canvas part.
         # But don't do this if the cursor is at the top/bottom edge so we can still scroll out
-        if self._old_cursor_coords is not None and self._old_cursor_coords != canv.cursor:
+        if self._old_cursor_coords is not None and self._old_cursor_coords != canv.cursor and canv.cursor != None:
             self._old_cursor_coords = None
             curscol, cursrow = canv.cursor
             if cursrow < self._trim_top:
@@ -411,7 +412,10 @@ class ScrollBar(urwid.WidgetDecoration):
         if not handled and hasattr(ow, 'set_scrollpos'):
             if button == 4:    # scroll wheel up
                 pos = ow.get_scrollpos(ow_size)
-                ow.set_scrollpos(pos - 1)
+                newpos = pos - 1
+                if newpos < 0:
+                    newpos = 0
+                ow.set_scrollpos(newpos)
                 return True
             elif button == 5:  # scroll wheel down
                 pos = ow.get_scrollpos(ow_size)

setup.py

@@ -23,6 +23,6 @@ setuptools.setup(
     entry_points= {
         'console_scripts': ['nomadnet=nomadnet.nomadnet:main']
     },
-    install_requires=['rns>=0.2.1', 'lxmf>=0.0.6', 'urwid>=2.1.2'],
+    install_requires=['rns>=0.2.4', 'lxmf>=0.0.9', 'urwid>=2.1.2'],
-    python_requires='>=3.5',
+    python_requires='>=3.6',
 )