diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index 1306fbc9..0d24c69d 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -69,6 +69,29 @@ If the Nix installation is in multi-user mode, don’t forget to restart the nix nix develop -c $SHELL ``` +## Regtest Environment + +For testing and development, CDK provides a complete regtest environment with Bitcoin, Lightning Network nodes, and CDK mints. + +### Quick Start +```bash +just regtest # Starts full environment with mprocs TUI +``` + +This provides: +- Bitcoin regtest node +- 4 Lightning Network nodes (2 CLN + 2 LND) +- 2 CDK mints (one connected to CLN, one to LND) +- Real-time log monitoring via mprocs +- Helper commands for testing Lightning payments and CDK operations + +### Comprehensive Guide +See [REGTEST_GUIDE.md](REGTEST_GUIDE.md) for complete documentation including: +- Detailed setup and usage instructions +- Development workflows and testing scenarios +- mprocs TUI interface guide +- Troubleshooting and advanced usage + ## Common Development Tasks ### Building the Project diff --git a/REGTEST_GUIDE.md b/REGTEST_GUIDE.md new file mode 100644 index 00000000..91958648 --- /dev/null +++ b/REGTEST_GUIDE.md @@ -0,0 +1,267 @@ +# CDK Regtest Environment Guide + +A comprehensive guide for setting up and using the CDK regtest environment for development and testing. + +## Quick Start + +### Start the Environment +```bash +# Start regtest with SQLite database (default) +just regtest + +# Or with REDB database +just regtest redb +``` + +The script will: +1. Check for `mprocs` and offer to install it if missing +2. Build necessary binaries +3. Set up Bitcoin regtest + 4 Lightning nodes + 2 CDK mints +4. Launch `mprocs` TUI showing all component logs +5. Both mints start automatically + +### Stop the Environment +Press `q` in mprocs or `Ctrl+C` in the terminal. Everything cleans up automatically. + +## Network Components + +When running, you get a complete Lightning Network environment: + +### Bitcoin Network +- **Bitcoin RPC**: `127.0.0.1:18443` (user: `testuser`, pass: `testpass`) + +### Lightning Nodes +- **CLN Node 1**: `$CDK_ITESTS_DIR/cln/one/regtest/lightning-rpc` +- **CLN Node 2**: `$CDK_ITESTS_DIR/cln/two/regtest/lightning-rpc` +- **LND Node 1**: `https://localhost:10009` +- **LND Node 2**: `https://localhost:10010` + +### CDK Mints +- **CLN Mint**: `http://127.0.0.1:8085` (connected to CLN node 1) +- **LND Mint**: `http://127.0.0.1:8087` (connected to LND node 2) + +### Environment Variables +Available in all terminals automatically: +- `CDK_TEST_MINT_URL`: CLN mint URL +- `CDK_TEST_MINT_URL_2`: LND mint URL +- `CDK_ITESTS_DIR`: Temporary directory with all data + +## Using the Environment + +All commands work from any terminal - they automatically find the running environment. + +### Lightning Node Operations +```bash +# Get node information +just ln-cln1 getinfo +just ln-cln2 getinfo +just ln-lnd1 getinfo +just ln-lnd2 getinfo + +# Create and pay invoices +just ln-cln1 invoice 1000 label "Test payment" +just ln-lnd1 payinvoice + +# Check balances and channels +just ln-cln1 listfunds +just ln-lnd1 listchannels +``` + +### Bitcoin Operations +```bash +just btc getblockchaininfo # Blockchain status +just btc getbalance # Wallet balance +just btc-mine 5 # Mine 5 blocks +``` + +### CDK Mint Operations +```bash +just mint-info # Show both mints' info +just mint-test # Run integration tests +just restart-mints # Recompile and restart mints +just regtest-status # Check all components +just regtest-logs # Show recent logs +``` + +## mprocs TUI Interface + +The `mprocs` interface shows all component logs in real-time: + +### Controls +- **Arrow keys**: Navigate between processes +- **Enter**: Focus on a process to see its output +- **Tab**: Switch between process list and output view +- **s**: Start a process (if stopped) +- **k**: Kill a process +- **r**: Restart a process +- **PageUp/PageDown**: Scroll through logs +- **?**: Show help +- **q**: Quit and stop environment + +### Process List +- `cln-mint`: CDK mint connected to CLN (auto-started) +- `lnd-mint`: CDK mint connected to LND (auto-started) +- `bitcoind`: Bitcoin regtest node logs +- `cln-one`: CLN node 1 logs +- `cln-two`: CLN node 2 logs +- `lnd-one`: LND node 1 logs +- `lnd-two`: LND node 2 logs + +## Development Workflows + +### Testing Lightning Payment Flow +```bash +# Terminal 1: Start environment +just regtest + +# Terminal 2: Create invoice and pay +just ln-cln1 invoice 1000 test "Test payment" +just ln-lnd1 payinvoice +just ln-cln1 listinvoices +just ln-lnd1 listpayments +``` + +### Developing Mint Code +```bash +# Terminal 1: Keep regtest running +just regtest + +# Terminal 2: After making code changes +just restart-mints # Recompiles and restarts both mints +just mint-info # Test the changes +just mint-test # Run integration tests +``` + +### Using CDK CLI Tools +```bash +# Terminal 1: Start environment +just regtest + +# Terminal 2: Use environment variables +cargo run --bin cdk-cli -- --mint-url $CDK_TEST_MINT_URL mint-info +cargo run --bin cdk-cli -- --mint-url $CDK_TEST_MINT_URL_2 mint-info +``` + +### Direct API Testing +```bash +# Query mint info directly +curl $CDK_TEST_MINT_URL/v1/info | jq +curl $CDK_TEST_MINT_URL/v1/keysets | jq + +# Test both mints +curl http://127.0.0.1:8085/v1/info | jq +curl http://127.0.0.1:8087/v1/info | jq +``` + +## File Structure + +All components run in a temporary directory: + +``` +$CDK_ITESTS_DIR/ +├── bitcoin/ # Bitcoin regtest data +├── cln/ +│ ├── one/ # CLN node 1 data +│ └── two/ # CLN node 2 data +├── lnd/ +│ ├── one/ # LND node 1 data +│ │ ├── tls.cert +│ │ └── data/chain/bitcoin/regtest/admin.macaroon +│ └── two/ # LND node 2 data +├── cln_mint/ # CLN mint working directory +├── lnd_mint/ # LND mint working directory +├── start_cln_mint.sh # Mint startup scripts +├── start_lnd_mint.sh +└── mprocs.yaml # mprocs configuration +``` + +## Installation Requirements + +### mprocs (TUI Interface) +If not installed, the script will offer to install it: +```bash +# Automatic installation during regtest setup +just regtest + +# Manual installation +cargo install mprocs + +# Or via package manager +# Ubuntu/Debian: apt install mprocs +# macOS: brew install mprocs +``` + +### System Dependencies +Managed automatically via Nix development shell: +- Bitcoin Core +- Core Lightning (CLN) +- LND (Lightning Network Daemon) +- Rust toolchain + +## Advanced Usage + +### Manual mprocs Launch +```bash +# If you need to restart just the mprocs interface +source /tmp/cdk_regtest_env +just regtest-logs +``` + +### Environment State +The environment creates a state file at `/tmp/cdk_regtest_env` that: +- Shares environment variables between terminals +- Allows `just` commands to work from anywhere +- Automatically cleaned up when environment stops + +### Process Management +From within mprocs: +- Restart individual mints after code changes +- Monitor specific component logs +- Start/stop services for testing scenarios + +## Troubleshooting + +### Environment Not Starting +- Check that ports are available: 8085, 8087, 18443, 19846, 19847, 10009, 10010 +- Ensure the Nix development shell is active: `nix develop` +- Check individual component logs in mprocs + +### Helper Commands Not Working +- Ensure the regtest environment is running +- Check that `/tmp/cdk_regtest_env` file exists +- Verify environment variables are set: `echo $CDK_TEST_MINT_URL` + +### Connection Issues +- Use `just regtest-status` to check component health +- Check mint logs with `just regtest-logs` +- Verify Lightning node status with `just ln-cln1 getinfo` + +### mprocs Issues +- If mprocs crashes, processes continue running +- Use `Ctrl+C` in the original terminal to clean up +- Restart with `just regtest-logs` + +## Common Error Solutions + +### "Port already in use" +```bash +# Find and kill processes using ports +sudo lsof -ti:8085 | xargs kill -9 +sudo lsof -ti:8087 | xargs kill -9 +``` + +### "Environment not found" +```bash +# Clean up and restart +rm -f /tmp/cdk_regtest_env +just regtest +``` + +### "Binary not found" +```bash +# Rebuild binaries +just build +just regtest +``` + +This environment provides everything needed for CDK development and testing in a single, easy-to-use interface! 🎉 diff --git a/crates/cdk-integration-tests/Cargo.toml b/crates/cdk-integration-tests/Cargo.toml index a824321a..26a9da98 100644 --- a/crates/cdk-integration-tests/Cargo.toml +++ b/crates/cdk-integration-tests/Cargo.toml @@ -35,7 +35,7 @@ uuid.workspace = true serde.workspace = true serde_json.workspace = true # ln-regtest-rs = { path = "../../../../ln-regtest-rs" } -ln-regtest-rs = { git = "https://github.com/thesimplekid/ln-regtest-rs", rev = "ed24716" } +ln-regtest-rs = { git = "https://github.com/thesimplekid/ln-regtest-rs", rev = "df81424" } lightning-invoice.workspace = true tracing.workspace = true tracing-subscriber.workspace = true diff --git a/flake.nix b/flake.nix index 311f5307..20def631 100644 --- a/flake.nix +++ b/flake.nix @@ -81,6 +81,7 @@ bitcoind sqlx-cli cargo-outdated + mprocs # Needed for github ci libz @@ -274,7 +275,7 @@ echo "Docker is available at $(which docker)" echo "Docker version: $(docker --version)" ''; - buildInputs = buildInputs ++ [ + buildInputs = buildInputs ++ [ stable_toolchain pkgs.docker-client ]; diff --git a/justfile b/justfile index 7e0cdf1c..21b085fa 100644 --- a/justfile +++ b/justfile @@ -153,6 +153,68 @@ nutshell-wallet-itest: #!/usr/bin/env bash ./misc/nutshell_wallet_itest.sh +# Start interactive regtest environment (Bitcoin + 4 LN nodes + 2 CDK mints) +regtest db="sqlite": + #!/usr/bin/env bash + ./misc/interactive_regtest_mprocs.sh {{db}} + +# Lightning Network Commands (require regtest environment to be running) + +# Get CLN node 1 info +ln-cln1 *ARGS: + #!/usr/bin/env bash + ./misc/regtest_helper.sh ln-cln1 {{ARGS}} + +# Get CLN node 2 info +ln-cln2 *ARGS: + #!/usr/bin/env bash + ./misc/regtest_helper.sh ln-cln2 {{ARGS}} + +# Get LND node 1 info +ln-lnd1 *ARGS: + #!/usr/bin/env bash + ./misc/regtest_helper.sh ln-lnd1 {{ARGS}} + +# Get LND node 2 info +ln-lnd2 *ARGS: + #!/usr/bin/env bash + ./misc/regtest_helper.sh ln-lnd2 {{ARGS}} + +# Bitcoin regtest commands +btc *ARGS: + #!/usr/bin/env bash + ./misc/regtest_helper.sh btc {{ARGS}} + +# Mine blocks in regtest +btc-mine blocks="10": + #!/usr/bin/env bash + ./misc/regtest_helper.sh btc-mine {{blocks}} + +# Show mint information +mint-info: + #!/usr/bin/env bash + ./misc/regtest_helper.sh mint-info + +# Run integration tests against regtest environment +mint-test: + #!/usr/bin/env bash + ./misc/regtest_helper.sh mint-test + +# Restart mints after recompiling (useful for development) +restart-mints: + #!/usr/bin/env bash + ./misc/regtest_helper.sh restart-mints + +# Show regtest environment status +regtest-status: + #!/usr/bin/env bash + ./misc/regtest_helper.sh show-status + +# Show regtest environment logs +regtest-logs: + #!/usr/bin/env bash + ./misc/regtest_helper.sh show-logs + run-examples: cargo r --example p2pk cargo r --example mint-token diff --git a/misc/interactive_regtest_mprocs.sh b/misc/interactive_regtest_mprocs.sh new file mode 100755 index 00000000..72aceef7 --- /dev/null +++ b/misc/interactive_regtest_mprocs.sh @@ -0,0 +1,320 @@ +#!/usr/bin/env bash + +# Interactive Regtest Environment for CDK with Direct Process Management +# This script sets up mprocs to manage the mint processes directly + +set -e + +# Function to wait for HTTP endpoint +wait_for_endpoint() { + local url=$1 + local timeout=${2:-60} + local start_time=$(date +%s) + + while true; do + local current_time=$(date +%s) + local elapsed_time=$((current_time - start_time)) + + if [ $elapsed_time -ge $timeout ]; then + echo "❌ Timeout waiting for $url" + return 1 + fi + + local http_status=$(curl -o /dev/null -s -w "%{http_code}" "$url" 2>/dev/null || echo "000") + + if [ "$http_status" -eq 200 ]; then + echo "✓ $url is ready" + return 0 + fi + + sleep 2 + done +} + +# Function to perform cleanup +cleanup() { + echo "Cleaning up..." + + # Remove state file for other sessions + rm -f "/tmp/cdk_regtest_env" + + if [ ! -z "$CDK_REGTEST_PID" ] && kill -0 $CDK_REGTEST_PID 2>/dev/null; then + echo "Killing the cdk regtest" + kill -2 $CDK_REGTEST_PID + wait $CDK_REGTEST_PID + fi + + echo "Environment terminated" + + # Remove the temporary directory + if [ ! -z "$CDK_ITESTS_DIR" ]; then + rm -rf "$CDK_ITESTS_DIR" + echo "Temp directory removed: $CDK_ITESTS_DIR" + fi + + # Unset all environment variables + unset CDK_ITESTS_DIR + unset CDK_ITESTS_MINT_ADDR + unset CDK_ITESTS_MINT_PORT_0 + unset CDK_ITESTS_MINT_PORT_1 + unset CDK_MINTD_DATABASE + unset CDK_TEST_MINT_URL + unset CDK_TEST_MINT_URL_2 + unset CDK_REGTEST_PID + unset RUST_BACKTRACE + unset CDK_TEST_REGTEST +} + +# Set up trap to call cleanup on script exit +trap cleanup EXIT + +export CDK_TEST_REGTEST=1 + +# Check for mprocs and offer to install if missing +if ! command -v mprocs >/dev/null 2>&1; then + echo "⚠️ mprocs not found - this tool is required for direct process management" + echo "Install it with: cargo install mprocs" + echo + read -p "Would you like to install mprocs now? (y/n): " -n 1 -r + echo + if [[ $REPLY =~ ^[Yy]$ ]]; then + echo "Installing mprocs..." + cargo install mprocs + if [ $? -eq 0 ]; then + echo "✓ mprocs installed successfully" + else + echo "❌ Failed to install mprocs." + exit 1 + fi + else + echo "❌ mprocs is required for this mode. Exiting." + exit 1 + fi + echo +fi + +# Parse command line arguments +CDK_MINTD_DATABASE=${1:-"sqlite"} # Default to sqlite if not specified + +# Create a temporary directory +export CDK_ITESTS_DIR=$(mktemp -d) +export CDK_ITESTS_MINT_ADDR="127.0.0.1" +export CDK_ITESTS_MINT_PORT_0=8085 +export CDK_ITESTS_MINT_PORT_1=8087 + +# Check if the temporary directory was created successfully +if [[ ! -d "$CDK_ITESTS_DIR" ]]; then + echo "Failed to create temp directory" + exit 1 +fi + +echo "==============================================" +echo "Starting CDK Regtest with Direct Process Management" +echo "==============================================" +echo "Temp directory: $CDK_ITESTS_DIR" +echo "Database type: $CDK_MINTD_DATABASE" +echo + +export CDK_MINTD_DATABASE="$CDK_MINTD_DATABASE" + +# Build the necessary binaries +echo "Building binaries..." +cargo build -p cdk-integration-tests --bin start_regtest +cargo build --bin cdk-mintd + +echo "Starting regtest network (Bitcoin + Lightning nodes)..." +cargo run --bin start_regtest & +export CDK_REGTEST_PID=$! + +# Create named pipe for progress tracking +mkfifo "$CDK_ITESTS_DIR/progress_pipe" +rm -f "$CDK_ITESTS_DIR/signal_received" + +# Start reading from pipe in background +(while read line; do + case "$line" in + "checkpoint1") + echo "✓ Regtest network is ready" + touch "$CDK_ITESTS_DIR/signal_received" + exit 0 + ;; + esac +done < "$CDK_ITESTS_DIR/progress_pipe") & + +# Wait for regtest setup (up to 120 seconds) +echo "Waiting for regtest network to be ready..." +for ((i=0; i<120; i++)); do + if [ -f "$CDK_ITESTS_DIR/signal_received" ]; then + break + fi + sleep 1 +done + +if [ ! -f "$CDK_ITESTS_DIR/signal_received" ]; then + echo "❌ Timeout waiting for regtest network" + exit 1 +fi + +# Create work directories for mints +mkdir -p "$CDK_ITESTS_DIR/cln_mint" +mkdir -p "$CDK_ITESTS_DIR/lnd_mint" + +# Set environment variables for easy access +export CDK_TEST_MINT_URL="http://$CDK_ITESTS_MINT_ADDR:$CDK_ITESTS_MINT_PORT_0" +export CDK_TEST_MINT_URL_2="http://$CDK_ITESTS_MINT_ADDR:$CDK_ITESTS_MINT_PORT_1" + +# Create state file for other terminal sessions +ENV_FILE="/tmp/cdk_regtest_env" +echo "export CDK_ITESTS_DIR=\"$CDK_ITESTS_DIR\"" > "$ENV_FILE" +echo "export CDK_TEST_MINT_URL=\"$CDK_TEST_MINT_URL\"" >> "$ENV_FILE" +echo "export CDK_TEST_MINT_URL_2=\"$CDK_TEST_MINT_URL_2\"" >> "$ENV_FILE" +echo "export CDK_REGTEST_PID=\"$CDK_REGTEST_PID\"" >> "$ENV_FILE" + +# Get the project root directory (where justfile is located) +PROJECT_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" + +# Create environment setup scripts for mprocs to use +cat > "$CDK_ITESTS_DIR/start_cln_mint.sh" << EOF +#!/usr/bin/env bash +cd "$PROJECT_ROOT" +export CDK_MINTD_CLN_RPC_PATH="$CDK_ITESTS_DIR/cln/one/regtest/lightning-rpc" +export CDK_MINTD_URL="http://127.0.0.1:8085" +export CDK_MINTD_WORK_DIR="$CDK_ITESTS_DIR/cln_mint" +export CDK_MINTD_LISTEN_HOST="127.0.0.1" +export CDK_MINTD_LISTEN_PORT=8085 +export CDK_MINTD_LN_BACKEND="cln" +export CDK_MINTD_MNEMONIC="eye survey guilt napkin crystal cup whisper salt luggage manage unveil loyal" +export RUST_BACKTRACE=1 +export CDK_MINTD_DATABASE="$CDK_MINTD_DATABASE" + +echo "Starting CLN Mint on port 8085..." +echo "Project root: $PROJECT_ROOT" +echo "Working directory: \$CDK_MINTD_WORK_DIR" +echo "CLN RPC path: \$CDK_MINTD_CLN_RPC_PATH" +echo "Database type: \$CDK_MINTD_DATABASE" +echo "---" + +exec cargo run --bin cdk-mintd +EOF + +cat > "$CDK_ITESTS_DIR/start_lnd_mint.sh" << EOF +#!/usr/bin/env bash +cd "$PROJECT_ROOT" +export CDK_MINTD_LND_ADDRESS="https://localhost:10010" +export CDK_MINTD_LND_CERT_FILE="$CDK_ITESTS_DIR/lnd/two/tls.cert" +export CDK_MINTD_LND_MACAROON_FILE="$CDK_ITESTS_DIR/lnd/two/data/chain/bitcoin/regtest/admin.macaroon" +export CDK_MINTD_URL="http://127.0.0.1:8087" +export CDK_MINTD_WORK_DIR="$CDK_ITESTS_DIR/lnd_mint" +export CDK_MINTD_LISTEN_HOST="127.0.0.1" +export CDK_MINTD_LISTEN_PORT=8087 +export CDK_MINTD_LN_BACKEND="lnd" +export CDK_MINTD_MNEMONIC="cattle gold bind busy sound reduce tone addict baby spend february strategy" +export RUST_BACKTRACE=1 +export CDK_MINTD_DATABASE="$CDK_MINTD_DATABASE" + +echo "Starting LND Mint on port 8087..." +echo "Project root: $PROJECT_ROOT" +echo "Working directory: \$CDK_MINTD_WORK_DIR" +echo "LND address: \$CDK_MINTD_LND_ADDRESS" +echo "Database type: \$CDK_MINTD_DATABASE" +echo "---" + +exec cargo run --bin cdk-mintd +EOF + +# Make scripts executable +chmod +x "$CDK_ITESTS_DIR/start_cln_mint.sh" +chmod +x "$CDK_ITESTS_DIR/start_lnd_mint.sh" + +echo +echo "==============================================" +echo "🎉 CDK Regtest Environment is Ready!" +echo "==============================================" +echo +echo "Network Information:" +echo " • Bitcoin RPC: 127.0.0.1:18443 (user: testuser, pass: testpass)" +echo " • CLN Node 1: $CDK_ITESTS_DIR/cln/one/regtest/lightning-rpc" +echo " • CLN Node 2: $CDK_ITESTS_DIR/cln/two/regtest/lightning-rpc" +echo " • LND Node 1: https://localhost:10009" +echo " • LND Node 2: https://localhost:10010" +echo +echo "CDK Mints (will be managed by mprocs):" +echo " • CLN Mint: $CDK_TEST_MINT_URL" +echo " • LND Mint: $CDK_TEST_MINT_URL_2" +echo +echo "Files and Directories:" +echo " • Working Directory: $CDK_ITESTS_DIR" +echo " • Start Scripts: $CDK_ITESTS_DIR/start_{cln,lnd}_mint.sh" +echo +echo "Environment Variables (available in other terminals):" +echo " • CDK_TEST_MINT_URL=\"$CDK_TEST_MINT_URL\"" +echo " • CDK_TEST_MINT_URL_2=\"$CDK_TEST_MINT_URL_2\"" +echo " • CDK_ITESTS_DIR=\"$CDK_ITESTS_DIR\"" +echo +echo "Starting mprocs with direct process management..." +echo +echo "In mprocs you can:" +echo " • 's' to start a process" +echo " • 'k' to kill a process" +echo " • 'r' to restart a process" +echo " • 'Enter' to focus on a process" +echo " • 'q' to quit and stop the environment" +echo "==============================================" + +# Wait a moment for everything to settle +sleep 2 + +# Create mprocs configuration with direct process management +MPROCS_CONFIG="$CDK_ITESTS_DIR/mprocs.yaml" +cat > "$MPROCS_CONFIG" << EOF +procs: + cln-mint: + shell: "$CDK_ITESTS_DIR/start_cln_mint.sh" + autostart: true + env: + CDK_ITESTS_DIR: "$CDK_ITESTS_DIR" + CDK_MINTD_DATABASE: "$CDK_MINTD_DATABASE" + + lnd-mint: + shell: "$CDK_ITESTS_DIR/start_lnd_mint.sh" + autostart: true + env: + CDK_ITESTS_DIR: "$CDK_ITESTS_DIR" + CDK_MINTD_DATABASE: "$CDK_MINTD_DATABASE" + + bitcoind: + shell: "while [ ! -f $CDK_ITESTS_DIR/bitcoin/regtest/debug.log ]; do sleep 1; done && tail -f $CDK_ITESTS_DIR/bitcoin/regtest/debug.log" + autostart: true + + cln-one: + shell: "while [ ! -f $CDK_ITESTS_DIR/cln/one/debug.log ]; do sleep 1; done && tail -f $CDK_ITESTS_DIR/cln/one/debug.log" + autostart: true + + cln-two: + shell: "while [ ! -f $CDK_ITESTS_DIR/cln/two/debug.log ]; do sleep 1; done && tail -f $CDK_ITESTS_DIR/cln/two/debug.log" + autostart: true + + lnd-one: + shell: "while [ ! -f $CDK_ITESTS_DIR/lnd/one/logs/bitcoin/regtest/lnd.log ]; do sleep 1; done && tail -f $CDK_ITESTS_DIR/lnd/one/logs/bitcoin/regtest/lnd.log" + autostart: true + + lnd-two: + shell: "while [ ! -f $CDK_ITESTS_DIR/lnd/two/logs/bitcoin/regtest/lnd.log ]; do sleep 1; done && tail -f $CDK_ITESTS_DIR/lnd/two/logs/bitcoin/regtest/lnd.log" + autostart: true + +settings: + mouse_scroll_speed: 3 + proc_list_width: 20 + hide_keymap_window: false + keymap_procs: + toggle_process: 's' + kill_process: 'k' + restart_process: 'r' + focus_process: 'Enter' + show_keymap: '?' +EOF + +# Start mprocs with direct process management +echo "Starting mprocs..." +cd "$CDK_ITESTS_DIR" +mprocs --config "$MPROCS_CONFIG" diff --git a/misc/regtest_helper.sh b/misc/regtest_helper.sh new file mode 100755 index 00000000..5e366825 --- /dev/null +++ b/misc/regtest_helper.sh @@ -0,0 +1,421 @@ +#!/usr/bin/env bash + +# Helper script for interacting with CDK regtest environment +# Run this after starting interactive_regtest_mprocs.sh + +# Check for environment state file first, then environment variable +ENV_FILE="/tmp/cdk_regtest_env" +if [ -f "$ENV_FILE" ]; then + source "$ENV_FILE" +elif [ ! -z "$CDK_ITESTS_DIR" ]; then + # Environment variable is set, create state file for other sessions + echo "export CDK_ITESTS_DIR=\"$CDK_ITESTS_DIR\"" > "$ENV_FILE" + echo "export CDK_TEST_MINT_URL=\"$CDK_TEST_MINT_URL\"" >> "$ENV_FILE" + echo "export CDK_TEST_MINT_URL_2=\"$CDK_TEST_MINT_URL_2\"" >> "$ENV_FILE" + echo "export CDK_MINTD_PID=\"$CDK_MINTD_PID\"" >> "$ENV_FILE" + echo "export CDK_MINTD_LND_PID=\"$CDK_MINTD_LND_PID\"" >> "$ENV_FILE" + echo "export CDK_REGTEST_PID=\"$CDK_REGTEST_PID\"" >> "$ENV_FILE" +else + echo "❌ CDK regtest environment not found!" + echo "Please run './misc/interactive_regtest_mprocs.sh' or 'just regtest' first" + exit 1 +fi + +# Validate that the environment is actually running +if [ -z "$CDK_ITESTS_DIR" ] || [ ! -d "$CDK_ITESTS_DIR" ]; then + echo "❌ CDK regtest environment not found or directory missing!" + echo "Please run './misc/interactive_regtest_mprocs.sh' or 'just regtest' first" + [ -f "$ENV_FILE" ] && rm "$ENV_FILE" # Clean up stale state file + exit 1 +fi + +show_help() { + echo "CDK Regtest Environment Helper" + echo "=============================" + echo + echo "Lightning Node Commands:" + echo " ln-cln1 - Execute command on CLN node 1" + echo " ln-cln2 - Execute command on CLN node 2" + echo " ln-lnd1 - Execute command on LND node 1" + echo " ln-lnd2 - Execute command on LND node 2" + echo + echo "Bitcoin Commands:" + echo " btc - Execute bitcoin-cli command" + echo " btc-mine [blocks] - Mine blocks (default: 10)" + echo + echo "CDK Mint Commands:" + echo " mint-info - Show mint information" + echo " mint-test - Run integration tests" + echo " restart-mints - Stop, recompile, and restart both mints (log mode)" + echo + echo "Environment Commands:" + echo " show-env - Show environment variables" + echo " show-logs - Show recent mint logs" + echo " show-status - Show status of all components" + echo " logs - Start mprocs TUI (adapts to current mode)" + echo + echo "Environment Modes:" + echo " just regtest - Log tailing mode (mints auto-start, logs to files)" + echo " just regtest-mprocs - Direct management (mprocs controls mint processes)" + echo + echo "Examples:" + echo " $0 ln-cln1 getinfo" + echo " $0 ln-lnd1 getinfo" + echo " $0 btc getblockcount" + echo " $0 btc-mine 5" + echo " $0 mint-info" + echo " $0 restart-mints # Only works in log tailing mode" + echo " $0 logs # Start mprocs viewer" +} + +# Bitcoin commands +btc_command() { + bitcoin-cli -regtest -rpcuser=testuser -rpcpassword=testpass -rpcport=18443 "$@" +} + +btc_mine() { + local blocks=${1:-10} + local address=$(btc_command getnewaddress) + btc_command generatetoaddress "$blocks" "$address" + echo "Mined $blocks blocks" +} + +# CLN commands +cln_command() { + local node=$1 + shift + lightning-cli --rpc-file="$CDK_ITESTS_DIR/cln/$node/regtest/lightning-rpc" "$@" +} + +# LND commands +lnd_command() { + local node=$1 + shift + local port + case $node in + "one") port=10009 ;; + "two") port=10010 ;; + *) echo "Unknown LND node: $node"; return 1 ;; + esac + + lncli --rpcserver=localhost:$port \ + --tlscertpath="$CDK_ITESTS_DIR/lnd/$node/tls.cert" \ + --macaroonpath="$CDK_ITESTS_DIR/lnd/$node/data/chain/bitcoin/regtest/admin.macaroon" \ + "$@" +} + +# Mint commands +mint_info() { + echo "CLN Mint (Port 8085):" + curl -s "$CDK_TEST_MINT_URL/v1/info" | jq . 2>/dev/null || curl -s "$CDK_TEST_MINT_URL/v1/info" + echo + echo "LND Mint (Port 8087):" + curl -s "$CDK_TEST_MINT_URL_2/v1/info" | jq . 2>/dev/null || curl -s "$CDK_TEST_MINT_URL_2/v1/info" +} + +mint_test() { + echo "Running integration tests..." + cargo test -p cdk-integration-tests +} + +# Environment info +show_env() { + echo "CDK Regtest Environment Variables:" + echo "=================================" + echo "CDK_ITESTS_DIR=$CDK_ITESTS_DIR" + echo "CDK_TEST_MINT_URL=$CDK_TEST_MINT_URL" + echo "CDK_TEST_MINT_URL_2=$CDK_TEST_MINT_URL_2" + echo "CDK_MINTD_PID=$CDK_MINTD_PID" + echo "CDK_MINTD_LND_PID=$CDK_MINTD_LND_PID" + echo "CDK_REGTEST_PID=$CDK_REGTEST_PID" +} + +show_logs() { + echo "=== Recent CLN Mint Logs ===" + if [ -f "$CDK_ITESTS_DIR/cln_mint/mintd.log" ]; then + tail -10 "$CDK_ITESTS_DIR/cln_mint/mintd.log" + else + echo "Log file not found" + fi + echo + echo "=== Recent LND Mint Logs ===" + if [ -f "$CDK_ITESTS_DIR/lnd_mint/mintd.log" ]; then + tail -10 "$CDK_ITESTS_DIR/lnd_mint/mintd.log" + else + echo "Log file not found" + fi +} + +start_mprocs() { + echo "Starting mprocs log viewer..." + + if ! command -v mprocs >/dev/null 2>&1; then + echo "❌ mprocs not found! Please install it with:" + echo " cargo install mprocs" + echo " or your package manager" + return 1 + fi + + # Check if we have the direct process management config + DIRECT_MPROCS_CONFIG="$CDK_ITESTS_DIR/mprocs.yaml" + FALLBACK_MPROCS_CONFIG="$CDK_ITESTS_DIR/mprocs_fallback.yaml" + + if [ -f "$DIRECT_MPROCS_CONFIG" ]; then + echo "Using direct process management mode..." + echo "In mprocs: 's' to start, 'k' to kill, 'r' to restart processes" + cd "$CDK_ITESTS_DIR" + mprocs --config "$DIRECT_MPROCS_CONFIG" + return + fi + + # Create fallback mprocs configuration for log tailing + cat > "$FALLBACK_MPROCS_CONFIG" << EOF +procs: + cln-mint: + shell: "touch $CDK_ITESTS_DIR/cln_mint/mintd.log && tail -f $CDK_ITESTS_DIR/cln_mint/mintd.log" + autostart: true + + lnd-mint: + shell: "touch $CDK_ITESTS_DIR/lnd_mint/mintd.log && tail -f $CDK_ITESTS_DIR/lnd_mint/mintd.log" + autostart: true + + bitcoind: + shell: "touch $CDK_ITESTS_DIR/bitcoin/regtest/debug.log && tail -f $CDK_ITESTS_DIR/bitcoin/regtest/debug.log" + autostart: true + + cln-one: + shell: "while [ ! -f $CDK_ITESTS_DIR/cln/one/regtest/log ]; do sleep 1; done && tail -f $CDK_ITESTS_DIR/cln/one/regtest/log" + autostart: true + + cln-two: + shell: "while [ ! -f $CDK_ITESTS_DIR/cln/two/regtest/log ]; do sleep 1; done && tail -f $CDK_ITESTS_DIR/cln/two/regtest/log" + autostart: true + + lnd-one: + shell: "while [ ! -f $CDK_ITESTS_DIR/lnd/one/logs/bitcoin/regtest/lnd.log ]; do sleep 1; done && tail -f $CDK_ITESTS_DIR/lnd/one/logs/bitcoin/regtest/lnd.log" + autostart: true + + lnd-two: + shell: "while [ ! -f $CDK_ITESTS_DIR/lnd/two/logs/bitcoin/regtest/lnd.log ]; do sleep 1; done && tail -f $CDK_ITESTS_DIR/lnd/two/logs/bitcoin/regtest/lnd.log" + autostart: true + +settings: + mouse_scroll_speed: 3 + proc_list_width: 20 + hide_keymap_window: false +EOF + + echo "Using log tailing mode..." + echo "Use 'q' to quit the log viewer" + cd "$CDK_ITESTS_DIR" + mprocs --config "$FALLBACK_MPROCS_CONFIG" +} + +show_status() { + echo "CDK Regtest Environment Status:" + echo "===============================" + + # Check processes + echo "Processes:" + if [ ! -z "$CDK_REGTEST_PID" ] && kill -0 $CDK_REGTEST_PID 2>/dev/null; then + echo " ✓ Regtest network (PID: $CDK_REGTEST_PID)" + else + echo " ❌ Regtest network" + fi + + if [ ! -z "$CDK_MINTD_PID" ] && kill -0 $CDK_MINTD_PID 2>/dev/null; then + echo " ✓ CLN Mint (PID: $CDK_MINTD_PID)" + else + echo " ❌ CLN Mint" + fi + + if [ ! -z "$CDK_MINTD_LND_PID" ] && kill -0 $CDK_MINTD_LND_PID 2>/dev/null; then + echo " ✓ LND Mint (PID: $CDK_MINTD_LND_PID)" + else + echo " ❌ LND Mint" + fi + + echo + echo "Network connectivity:" + if curl -s "$CDK_TEST_MINT_URL/v1/info" >/dev/null 2>&1; then + echo " ✓ CLN Mint responding" + else + echo " ❌ CLN Mint not responding" + fi + + if curl -s "$CDK_TEST_MINT_URL_2/v1/info" >/dev/null 2>&1; then + echo " ✓ LND Mint responding" + else + echo " ❌ LND Mint not responding" + fi +} + +restart_mints() { + echo "===============================" + echo "Restarting CDK Mints" + echo "===============================" + + # Stop existing mints + echo "Stopping existing mints..." + if [ ! -z "$CDK_MINTD_PID" ] && kill -0 $CDK_MINTD_PID 2>/dev/null; then + echo " Stopping CLN Mint (PID: $CDK_MINTD_PID)" + kill -2 $CDK_MINTD_PID + wait $CDK_MINTD_PID 2>/dev/null || true + fi + + if [ ! -z "$CDK_MINTD_LND_PID" ] && kill -0 $CDK_MINTD_LND_PID 2>/dev/null; then + echo " Stopping LND Mint (PID: $CDK_MINTD_LND_PID)" + kill -2 $CDK_MINTD_LND_PID + wait $CDK_MINTD_LND_PID 2>/dev/null || true + fi + + # Recompile + echo "Recompiling cdk-mintd..." + if ! cargo build --bin cdk-mintd; then + echo "❌ Compilation failed" + return 1 + fi + echo "✓ Compilation successful" + + # Restart CLN mint + echo "Starting CLN Mint..." + export CDK_MINTD_CLN_RPC_PATH="$CDK_ITESTS_DIR/cln/one/regtest/lightning-rpc" + export CDK_MINTD_URL="http://127.0.0.1:8085" + export CDK_MINTD_WORK_DIR="$CDK_ITESTS_DIR/cln_mint" + export CDK_MINTD_LISTEN_HOST="127.0.0.1" + export CDK_MINTD_LISTEN_PORT=8085 + export CDK_MINTD_LN_BACKEND="cln" + export CDK_MINTD_MNEMONIC="eye survey guilt napkin crystal cup whisper salt luggage manage unveil loyal" + export RUST_BACKTRACE=1 + + cargo run --bin cdk-mintd > "$CDK_MINTD_WORK_DIR/mintd.log" 2>&1 & + NEW_CLN_PID=$! + + # Wait for CLN mint to be ready + echo "Waiting for CLN mint to start..." + local start_time=$(date +%s) + while true; do + local current_time=$(date +%s) + local elapsed_time=$((current_time - start_time)) + + if [ $elapsed_time -ge 30 ]; then + echo "❌ Timeout waiting for CLN mint" + return 1 + fi + + if curl -s "http://127.0.0.1:8085/v1/info" >/dev/null 2>&1; then + echo "✓ CLN Mint ready" + break + fi + sleep 1 + done + + # Restart LND mint + echo "Starting LND Mint..." + export CDK_MINTD_LND_ADDRESS="https://localhost:10010" + export CDK_MINTD_LND_CERT_FILE="$CDK_ITESTS_DIR/lnd/two/tls.cert" + export CDK_MINTD_LND_MACAROON_FILE="$CDK_ITESTS_DIR/lnd/two/data/chain/bitcoin/regtest/admin.macaroon" + export CDK_MINTD_URL="http://127.0.0.1:8087" + export CDK_MINTD_WORK_DIR="$CDK_ITESTS_DIR/lnd_mint" + export CDK_MINTD_LISTEN_HOST="127.0.0.1" + export CDK_MINTD_LISTEN_PORT=8087 + export CDK_MINTD_LN_BACKEND="lnd" + export CDK_MINTD_MNEMONIC="cattle gold bind busy sound reduce tone addict baby spend february strategy" + + cargo run --bin cdk-mintd > "$CDK_MINTD_WORK_DIR/mintd.log" 2>&1 & + NEW_LND_PID=$! + + # Wait for LND mint to be ready + echo "Waiting for LND mint to start..." + start_time=$(date +%s) + while true; do + current_time=$(date +%s) + elapsed_time=$((current_time - start_time)) + + if [ $elapsed_time -ge 30 ]; then + echo "❌ Timeout waiting for LND mint" + return 1 + fi + + if curl -s "http://127.0.0.1:8087/v1/info" >/dev/null 2>&1; then + echo "✓ LND Mint ready" + break + fi + sleep 1 + done + + # Update PIDs in state file + CDK_MINTD_PID=$NEW_CLN_PID + CDK_MINTD_LND_PID=$NEW_LND_PID + + # Update state file + echo "export CDK_ITESTS_DIR=\"$CDK_ITESTS_DIR\"" > "$ENV_FILE" + echo "export CDK_TEST_MINT_URL=\"$CDK_TEST_MINT_URL\"" >> "$ENV_FILE" + echo "export CDK_TEST_MINT_URL_2=\"$CDK_TEST_MINT_URL_2\"" >> "$ENV_FILE" + echo "export CDK_MINTD_PID=\"$CDK_MINTD_PID\"" >> "$ENV_FILE" + echo "export CDK_MINTD_LND_PID=\"$CDK_MINTD_LND_PID\"" >> "$ENV_FILE" + echo "export CDK_REGTEST_PID=\"$CDK_REGTEST_PID\"" >> "$ENV_FILE" + + echo + echo "✅ Mints restarted successfully!" + echo " CLN Mint: http://127.0.0.1:8085 (PID: $CDK_MINTD_PID)" + echo " LND Mint: http://127.0.0.1:8087 (PID: $CDK_MINTD_LND_PID)" + echo "===============================" +} + +# Main command dispatcher +case "$1" in + "ln-cln1") + shift + cln_command "one" "$@" + ;; + "ln-cln2") + shift + cln_command "two" "$@" + ;; + "ln-lnd1") + shift + lnd_command "one" "$@" + ;; + "ln-lnd2") + shift + lnd_command "two" "$@" + ;; + "btc") + shift + btc_command "$@" + ;; + "btc-mine") + shift + btc_mine "$@" + ;; + "mint-info") + mint_info + ;; + "mint-test") + mint_test + ;; + "restart-mints") + restart_mints + ;; + "show-env") + show_env + ;; + "show-logs") + show_logs + ;; + "show-status") + show_status + ;; + "logs") + start_mprocs + ;; + "help"|"-h"|"--help"|"") + show_help + ;; + *) + echo "Unknown command: $1" + echo "Run '$0 help' for available commands" + exit 1 + ;; +esac