Merge pull request #47 from hummingbot/feat/base-page-class

Feat/base page class

INSTALLATION.md

@@ -0,0 +1,70 @@
+## Requirements
+
+You will need to install [StreamLit](https://streamlit.io/). For information about Streamlit installation, see the instructions located at https://docs.streamlit.io/library/get-started/installation.
+
+You will also need to install either [Anaconda](https://www.anaconda.com/) or [Miniconda](https://docs.conda.io/en/latest/miniconda.html) to get Conda:
+* [Anaconda](https://www.anaconda.com/) is a comprehensive Python distribution that includes a large number of pre-installed data science libraries and packages. It is designed to be an all-in-one solution for data science and machine learning tasks. When you install Anaconda, it comes with a collection of popular Python packages like NumPy, pandas, matplotlib, scikit-learn, and more. 
+* [Miniconda](https://docs.conda.io/en/latest/miniconda.html) is a minimal version of Anaconda. It includes only the essential components, such as Python interpreter and Conda package manager. Unlike Anaconda, Miniconda doesn't come with pre-installed packages, which makes its download size much smaller. 
+
+This repository is maintained by Hummingbot Foundation as a companion for users of [Hummingbot](https://github.com/hummingbot/hummingbot), the open source framework for building high-frequency crypto trading bots.
+
+Watch this video to understand how it works:
+https://www.loom.com/share/72d05bcbaf4048a399e3f9247d756a63
+
+## Installation
+
+1. Install Steamlit and Conda packages utilizing their instructions for your specific environment:
+* Install  [StreamLit](https://docs.streamlit.io/library/get-started/installation)
+* Install [Anaconda](https://docs.anaconda.com/free/anaconda/install/index.html) or [Miniconda](https://docs.conda.io/en/latest/miniconda.html)
+
+2. Clone this repo and navigate to the created directory
+```bash
+git clone https://github.com/hummingbot/dashboard.git
+cd dashboard
+```
+
+3. Run conda command to create an isolated `conda` environment and install dependencies
+```
+conda env create -f environment_conda.yml
+```
+
+4. Activate the isoldated 'conda' environment
+```bash
+conda activate dashboard
+```
+
+5. Run the app
+```bash
+streamlit run main.py
+```
+
+## Data Feed
+
+Your `dashboard` environment needs to have access to the database for your Hummingbot environment. This is done by setting up a symbolic link to the 'data' directory of your running Hummingbot instance. 
+
+The data directory differs for Docker versus Source installed Hummingbot. Data directory for each is as follows:
+* Docker installed: /path/to/hummingbot/hummingbot_files/data
+* Source installed: /path/to/hummingbot/data
+
+
+Create a symlink to your Hummingbot `/data` directory
+```bash
+# replace `/path/to/hummingbotdata` with the actual path
+ln -s /path/to/hummingbotdata data
+
+# if you need to remove the symlink
+unlink data
+```
+
+## Updating Dependencies
+
+To update the `dashboard` environment for new dependencies, run:
+```
+conda env update -f environment_conda.yml
+```
+
+To updated the `dashboard` source for latest version, run:
+```
+cd dashboard
+git pull
+```

README.md

@@ -1,129 +1,31 @@
-# Hummingbot Dashboard
+Hummingbot Dashboard is a community project to build dashboards that help you deploy, manage, backtest, and analyze Hummingbot instances (and much more!) Each dashboard is maintained by 1-2 community members.
 
-Collection of data visualization and analysis Hummingbot-related dashboards. The dashboards helps you run and manage Hummingbot, analyze performance, analyze trade data, and much more!
-
-Dashboard is built using [StreamLit](https://streamlit.io/) and uses the Conda environment & package manager to simiplify installation, updates, and manage dependencies. 
-
-You will need to install [StreamLit](https://streamlit.io/). For information about Streamlit installation, see the instructions located at https://docs.streamlit.io/library/get-started/installation.
-
-You will also need to install either [Anaconda](https://www.anaconda.com/) or [Miniconda](https://docs.conda.io/en/latest/miniconda.html) to get Conda:
-* [Anaconda](https://www.anaconda.com/) is a comprehensive Python distribution that includes a large number of pre-installed data science libraries and packages. It is designed to be an all-in-one solution for data science and machine learning tasks. When you install Anaconda, it comes with a collection of popular Python packages like NumPy, pandas, matplotlib, scikit-learn, and more. 
-* [Miniconda](https://docs.conda.io/en/latest/miniconda.html) is a minimal version of Anaconda. It includes only the essential components, such as Python interpreter and Conda package manager. Unlike Anaconda, Miniconda doesn't come with pre-installed packages, which makes its download size much smaller. 
-
-This repository is maintained by Hummingbot Foundation as a companion for users of [Hummingbot](https://github.com/hummingbot/hummingbot), the open source framework for building high-frequency crypto trading bots.
-
-Watch this video to understand how it works:
-https://www.loom.com/share/72d05bcbaf4048a399e3f9247d756a63
-
-
-
-## Dashboards
+### Dashboards
 
 Here are the current dashboards in the collection:
 
-### 🚀 Strategy Performance (WIP)
+* 🚀 **Strategy Performance**: Analyze the performance of a running Hummingbot instance
+* 🐙 **Bot Orchestration**: Deploy and manage Hummingbot instances
+* ⚙️ **Backtest Manager**: Deploy and manage backtests of directional strategies
+* 🗂 **Candles Downloader**: Download historical exchange data as OHLVC candles. Supports multiple trading pairs and custom time ranges/intervals.
+* 🔍 **DB Inspector**: Inspect and analyze the orders and trades data contained in a Hummingbot strategy database
+* 🧙 **Token Spreads**: Identify cross-exchange trading opportunities by analyzing differences in token spreads across venues
+* 🦉 **TVL vs MCAPs**: Easily compare various DeFi protocols based on their market capitalization and total value locked, using DeFiLlama data.
 
-Dashboard that helps you analyze the performance of a running Hummingbot instance
+### Installation
 
-### 🧙 XE Token Analyzer
+This project is built using [StreamLit](https://streamlit.io/) and uses Anaconda and Docker to simplify installation, updates, and manage dependencies. 
 
-Dashboard that helps you visualize the bid-ask spread and volume of different tokens across the crypto exchange landscape. 
+See [Installation](https://github.com/hummingbot/dashboard/blob/feat/base-page-class/INSTALLATION.md) for how to install and update the dashboard.
 
-This app is most helpful for Hummingbot users running the [Cross-Exchange Market Making](https://hummingbot.org/strategies/cross-exchange-market-making/) and [Arbitrage](https://hummingbot.org/strategies/arbitrage/) strategies.
+### Contributions
 
-### 🧳 Hummingbot DB
+We welcome contributions from the community! See [Contribution](https://github.com/hummingbot/dashboard/blob/feat/base-page-class/CONTRIBUTING.md) for more information.
 
-Inspect and analyze the orders and trades data contained in a SQLite database for a strategy or script. 
-
-These files are located in the `/data` folder in Hummingbot, and are named `<strategy_name>.sqlite`.
-
-### 🦉 TVL vs MCAP Analysis
-
-Easily compare various DeFi protocols based on their market capitalization and total value locked, using DeFiLlama data.
-
-### 🗂 Candles Downloader
-
-Download historical exchange data as OHLVC candles. Supports multiple trading pairs and custom time ranges/intervals.
-
-Current Hummingbot connectors supported:
-* `binance`
-* `binance_perpetual`
-
-### 📋 Data
-
-Reference data for the various apps this collection.
-
-## Installation
-
-1. Install Steamlit and Conda packages utilizing their instructions for your specific environment:
-* Install  [StreamLit](https://docs.streamlit.io/library/get-started/installation)
-* Install [Anaconda](https://docs.anaconda.com/free/anaconda/install/index.html) or [Miniconda](https://docs.conda.io/en/latest/miniconda.html)
-
-2. Clone this repo and navigate to the created directory
-```bash
-git clone https://github.com/hummingbot/dashboard.git
-cd dashboard
-```
-
-3. Run conda command to create an isolated `conda` environment and install dependencies
-```
-conda env create -f environment_conda.yml
-```
-
-4. Activate the isoldated 'conda' environment
-```bash
-conda activate dashboard
-```
-
-5. Run the app
-```bash
-streamlit run main.py
-```
-
-## Data Feed
-
-Your `dashboard` environment needs to have access to the database for your Hummingbot environment. This is done by setting up a symbolic link to the 'data' directory of your running Hummingbot instance. 
-
-The data directory differs for Docker versus Source installed Hummingbot. Data directory for each is as follows:
-* Docker installed: /path/to/hummingbot/hummingbot_files/data
-* Source installed: /path/to/hummingbot/data
-
-
-Create a symlink to your Hummingbot `/data` directory
-```bash
-# replace `/path/to/hummingbotdata` with the actual path
-ln -s /path/to/hummingbotdata data
-
-# if you need to remove the symlink
-unlink data
-```
-
-
-
-## Updates
-
-To update the `dashboard` environment for new dependencies, run:
-```
-conda env update -f environment_conda.yml
-```
-
-To updated the `dashboard` source for latest version, run:
-```
-cd dashboard
-git pull
-```
-
-## Contributions
-
-We welcome new data dashboards, bug fixes, and improvements by the community!
-
-To submit a contribution, fork a clone of repository, add or make changes, and issue a pull request. See general guidelines for contributing to Hummingbot listed at https://hummingbot.org/developers/contributions.
-
-
-## Participation
+### Meetings
 
 We hold bi-weekly livestream Dashboard project meetings. You can participate on our [Discord](https://discord.gg/hummingbot) 
-* Alternating Thursdays, 3pm GMT / 11am EST / 8am PST / 11pm SIN
+* Alternating Wenesdays, 3pm GMT / 11am EST / 8am PST / 11pm SIN
 * Design, Status, Demos, etc
 
 

environment_conda.yml

@@ -19,6 +19,8 @@ dependencies:
       - jupyter
       - optuna
       - optuna-dashboard
+      - pathlib
       - streamlit-ace
+      - st-pages
       - git+https://github.com/hummingbot/hbot-remote-client-py.git
       - git+https://github.com/hummingbot/docker-manager.git

main.py

@@ -1,19 +1,27 @@
 import streamlit as st
+from st_pages import Page, show_pages
 
-apptitle = "High Frequency Trading"
-st.set_page_config(page_title=apptitle, page_icon="🦅", layout="wide")
+from utils.st_utils import initialize_st_page
+
+
+initialize_st_page(title="Hummingbot Dashboard", icon="📊")
+
+show_pages(
+    [
+        Page("main.py", "Hummingbot Dashboard", "📊"),
+        Page("pages/strategy_performance/app.py", "Strategy Performance", "🚀"),
+        Page("pages/bot_orchestration/app.py", "Bot Orchestration", "🐙"),
+        Page("pages/backtest_manager/app.py", "Backtest Manager", "⚙️"),
+        Page("pages/candles_downloader/app.py", "Candles Downloader", "🗂"),
+        Page("pages/db_inspector/app.py", "DB Inspector", "🔍"),
+        Page("pages/token_spreads/app.py", "Token Spreads", "🧙"),
+        Page("pages/tvl_vs_mcap/app.py", "TVL vs Market Cap", "🦉"),
+    ]
+)
 
-st.title("Welcome!")
-st.write("---")
-st.code("💡 The purpose of this dashboard is to provide useful information for high frequency trading traders")
-st.write("")
 st.write("Watch this video to understand how the dashboard works! 🦅")
 c1, c2, c3 = st.columns([1, 6, 1])
 with c2:
     st.video("https://youtu.be/l6PWbN2pDK8")
 st.write("If you want to contribute, post your idea in #dev-channel of [hummingbot discord](https://discord.gg/CjxZtkrH)")
 st.write("---")
-
-
-
-

pages/backtest_manager/README.md

@@ -0,0 +1 @@
+Deploy and manage backtests of directional strategies

pages/backtest_manager/app.py

@@ -12,16 +12,15 @@ from utils.file_templates import strategy_optimization_template, directional_str
 from utils.os_utils import load_directional_strategies, save_file, get_function_from_file
 import optuna
 
-st.set_page_config(
-    page_title="Hummingbot Dashboard",
-    page_icon="🚀",
-    layout="wide",
-)
+from utils.st_utils import initialize_st_page
+
+
+initialize_st_page(title="Backtest Manager", icon="⚙️")
+
+# Start content here
 if "strategy_params" not in st.session_state:
     st.session_state.strategy_params = {}
 
-st.title("⚙️ Backtesting")
-
 create, modify, backtest, optimize, analyze = st.tabs(["Create", "Modify", "Backtest", "Optimize", "Analyze"])
 
 with create:

pages/bot_orchestration/README.md

@@ -0,0 +1 @@
+Deploy and manage Hummingbot instances

pages/bot_orchestration/app.py

@@ -1,21 +1,17 @@
 import time
-
-import pandas as pd
-from hbotrc import BotCommands
-import streamlit as st
-from docker_manager import DockerManager
-
 import constants
+import pandas as pd
+import streamlit as st
 
-st.set_page_config(
-    page_title="Hummingbot Dashboard",
-    page_icon=":bar_chart:",
-    layout="wide",
-    initial_sidebar_state="collapsed"
-)
-st.title("🐙 Bot Orchestration")
-st.write("---")
+from docker_manager import DockerManager
+from hbotrc import BotCommands
 
+from utils.st_utils import initialize_st_page
+
+
+initialize_st_page(title="Bot Orchestration", icon="🐙")
+
+# Start content here
 docker_manager = DockerManager()
 
 active_containers = docker_manager.get_active_containers()

pages/candles_downloader/README.md

@@ -0,0 +1 @@
+Download historical exchange data as OHLVC candles. Supports multiple trading pairs and custom time ranges/intervals.

pages/candles_downloader/app.py

@@ -7,14 +7,13 @@ import constants
 from utils import os_utils
 from docker_manager import DockerManager
 
-st.set_page_config(
-    page_title="Candles Downloader",
-    page_icon=":bar_chart:",
-    layout="wide",
-)
+from utils.st_utils import initialize_st_page
+
+
+initialize_st_page(title="Candles Downloader", icon="🗂️")
+
+# Start content here
 docker_manager = DockerManager()
-st.write(f"# 🗂️ Candles Downloader")
-st.write("---")
 
 c1, c2, c3 = st.columns([2, 2, 0.5])
 with c1:

pages/db_inspector/README.md

@@ -0,0 +1 @@
+Inspect and analyze the orders and trades data contained in a Hummingbot strategy database

pages/db_inspector/app.py

@@ -3,6 +3,12 @@ import streamlit as st
 import sqlite3
 import pandas as pd
 
+from utils.st_utils import initialize_st_page
+
+
+initialize_st_page(title="DB Inspector", icon="🔍")
+
+# Start content here
 @st.cache_data
 def get_table_data(database_name: str, table_name: str):
     conn = sqlite3.connect(database_name)
@@ -17,9 +23,6 @@ def get_all_tables(database_name: str):
     tables = [table_row[0] for table_row in cursor.fetchall()]
     return tables
 
-st.set_page_config(layout='wide')
-st.title("🧳 Hummingbot Database Analyzer")
-st.write("---")
 uploaded_file = st.file_uploader("Add your database")
 
 if uploaded_file is not None:

pages/reference_data/7_📋_Data.py

@@ -3,30 +3,30 @@ import streamlit as st
 import CONFIG
 from utils.coingecko_utils import CoinGeckoUtils
 from utils.miner_utils import MinerUtils
+from utils.st_utils import initialize_st_page
+
 
 @st.cache_data
 def get_all_coins_df():
     return CoinGeckoUtils().get_all_coins_df()
 
+
 @st.cache_data
 def get_all_exchanges_df():
     return CoinGeckoUtils().get_all_exchanges_df()
 
+
 @st.cache_data
 def get_miner_stats_df():
     return MinerUtils().get_miner_stats_df()
 
+
 @st.cache_data
 def get_coin_tickers_by_id_list(coins_id: list):
     return CoinGeckoUtils().get_coin_tickers_by_id_list(coins_id)
 
-st.set_page_config(
-    page_title="Crypto Data",
-    page_icon=":bar_chart:",
-    layout="wide",
-    initial_sidebar_state="collapsed"
-)
-st.title("📋 Data Available")
+
+initialize_st_page(title="Crypto Data", icon=":bar_chart:")
 
 with st.spinner(text='In progress'):
     exchanges_df = get_all_exchanges_df()

pages/strategy_performance/README.md

@@ -0,0 +1 @@
+This page helps you load the database file of a Hummingbot strategy and analyze its performance.

pages/strategy_performance/app.py

@@ -1,19 +1,15 @@
 import os
-
 import pandas as pd
 import streamlit as st
 
 from utils.database_manager import DatabaseManager
 from utils.graphs import CandlesGraph
+from utils.st_utils import initialize_st_page
 
-st.set_page_config(
-    page_title="Hummingbot Dashboard",
-    page_icon=":bar_chart:",
-    layout="wide",
-    initial_sidebar_state="collapsed"
-)
-st.title("🚀 Strategy Performance")
 
+initialize_st_page(title="Strategy Performance", icon="🚀")
+
+# Start content here
 intervals = {
     "1m": 60,
     "3m": 60 * 3,
@@ -25,6 +21,7 @@ intervals = {
     "1d": 60 * 60 * 24,
 }
 
+
 @st.cache_resource
 def get_database(db_name: str):
     db_manager = DatabaseManager(db_name)

pages/token_spreads/README.md

@@ -0,0 +1 @@
+Identify cross-exchange trading opportunities by analyzing differences in token spreads across venues

pages/token_spreads/app.py

@@ -1,10 +1,15 @@
-import pandas as pd
 import streamlit as st
+from pathlib import Path
 import plotly.express as px
 import CONFIG
 from utils.coingecko_utils import CoinGeckoUtils
 from utils.miner_utils import MinerUtils
+from utils.st_utils import initialize_st_page
 
+
+initialize_st_page(title="Token Spreads", icon="🧙")
+
+# Start content here
 cg_utils = CoinGeckoUtils()
 miner_utils = MinerUtils()
 
@@ -24,10 +29,6 @@ def get_miner_stats_df():
 def get_coin_tickers_by_id_list(coins_id: list):
     return cg_utils.get_coin_tickers_by_id_list(coins_id)
 
-st.set_page_config(layout='wide')
-st.title("🧙‍Cross Exchange Token Analyzer")
-st.write("---")
-
 with st.spinner(text='In progress'):
     exchanges_df = get_all_exchanges_df()
     coins_df = get_all_coins_df()
@@ -35,7 +36,6 @@ with st.spinner(text='In progress'):
 
 miner_coins = coins_df.loc[coins_df["symbol"].isin(miner_stats_df["base"].str.lower().unique()), "name"]
 
-st.write("### Coins filter 🦅")
 tokens = st.multiselect(
     "Select the tokens to analyze:",
     options=coins_df["name"],
@@ -47,8 +47,7 @@ coins_id = coins_df.loc[coins_df["name"].isin(tokens), "id"].tolist()
 coin_tickers_df = get_coin_tickers_by_id_list(coins_id)
 coin_tickers_df["coin_name"] = coin_tickers_df.apply(lambda x: coins_df.loc[coins_df["id"] == x.token_id, "name"].item(), axis=1)
 
-st.sidebar.write("### Exchanges filter 🦅")
-exchanges = st.sidebar.multiselect(
+exchanges = st.multiselect(
     "Select the exchanges to analyze:",
     options=exchanges_df["name"],
     default=[exchange for exchange in CONFIG.MINER_EXCHANGES if exchange in exchanges_df["name"].unique()]
@@ -75,6 +74,6 @@ fig = px.scatter(
     }
 )
 
-st.sidebar.write("# Data filters 🏷")
-st.sidebar.code("🧳 New filters coming. \nReach us on discord \nif you want to propose one!")
+# st.write("# Data filters 🏷")
+# st.code("🧳 New filters coming. \nReach us on discord \nif you want to propose one!")
 st.plotly_chart(fig, use_container_width=True)

pages/tvl_vs_mcap/README.md

@@ -0,0 +1,3 @@
+Easily compare various DeFi protocols based on their market capitalization and total value locked, using DeFiLlama data.
+
+Data Source: [DefiLlama](https://defillama.com/)

pages/tvl_vs_mcap/app.py

@@ -4,6 +4,12 @@ import pandas as pd
 import plotly.express as px
 from defillama import DefiLlama
 
+from utils.st_utils import initialize_st_page
+
+
+initialize_st_page(title="TVL vs Market Cap", icon="🦉")
+
+# Start content here
 MIN_TVL = 1000000.
 MIN_MCAP = 1000000.
 
@@ -17,18 +23,13 @@ def get_tvl_mcap_data():
 def get_protocols_by_chain_category(protocols: pd.DataFrame, group_by: list, nth: list):
     return protocols.sort_values('tvl', ascending=False).groupby(group_by).nth(nth).reset_index()
 
-st.set_page_config(layout='wide')
-st.title("🦉 TVL vs MCAP Analysis")
-st.write("---")
-st.code("💡 Source: [DefiLlama](https://defillama.com/)")
-
 with st.spinner(text='In progress'):
     tvl_mcap_df = get_tvl_mcap_data()
 
 default_chains = ["Ethereum", "Solana", "Binance", "Polygon", "Multi-Chain", "Avalanche"]
 
-st.sidebar.write("### Chains filter 🔗")
-chains = st.sidebar.multiselect(
+st.write("### Chains 🔗")
+chains = st.multiselect(
     "Select the chains to analyze:",
     options=tvl_mcap_df["chain"].unique(),
     default=default_chains)
@@ -52,10 +53,10 @@ scatter = px.scatter(
 
 st.plotly_chart(scatter, use_container_width=True)
 
-st.sidebar.write("---")
-st.sidebar.write("### SunBurst filter 🔗")
-groupby = st.sidebar.selectbox('Group by:', [['chain', 'category'], ['category', 'chain']])
-nth = st.sidebar.slider('Top protocols by Category', min_value=1, max_value=5)
+st.write("---")
+st.write("### SunBurst 🌞")
+groupby = st.selectbox('Group by:', [['chain', 'category'], ['category', 'chain']])
+nth = st.slider('Top protocols by Category', min_value=1, max_value=5)
 
 proto_agg = get_protocols_by_chain_category(tvl_mcap_df[tvl_mcap_df["chain"].isin(chains)], groupby, np.arange(0, nth, 1).tolist())
 groupby.append("slug")
@@ -67,7 +68,4 @@ sunburst = px.sunburst(
     title="SunBurst",
     template="plotly_dark",)
 
-st.plotly_chart(sunburst, use_container_width=True)
-
-st.sidebar.write("# Data filters 🏷")
-st.sidebar.code("🧳 New filters coming. \nReach us on discord \nif you want to propose one!")
+st.plotly_chart(sunburst, use_container_width=True)

utils/st_utils.py

@@ -0,0 +1,20 @@
+import os.path
+from pathlib import Path
+import inspect
+
+import streamlit as st
+
+
+def initialize_st_page(title: str, icon: str, layout="wide"):
+    st.set_page_config(
+        page_title=title,
+        page_icon=icon,
+        layout=layout,
+    )
+    st.title(f"{icon} {title}")
+    caller_frame = inspect.currentframe().f_back
+
+    current_directory = Path(os.path.dirname(inspect.getframeinfo(caller_frame).filename))
+    readme_path = current_directory / "README.md"
+    with st.expander("About This Page"):
+        st.write(readme_path.read_text())