Skip to content

Running a Nomad Node

Difficulty: Medium (linux server management, cli usage)

Setup Time: ~1 hr

Technical Requirements

  • Dedicated server or virtual machine supporting Intel SGX2, for example:
  • Linux Kernel >=5.11
  • Docker (with compose)
  • Intel Aesmd

SGX Setup (Ubuntu 24.04 LTS)

  1. Ensure SGX is enabled in BIOS

Note: On OVH, make sure to enable Remote KVM or IPMI when provisioning the machine to be able to access the BIOS

  1. SSH into your server
ssh ubuntu@my.server.ip

Note: It is highly recommended to do some basic hardening on the instance.

  • Create a new user and delete the ubuntu user
  • Setup ~/.ssh/authorized_keys and disable password login for ssh
  • Routinely update the system packages
  1. Setup Intel's package signing key and install aesmd
curl -fsSL https://download.01.org/intel-sgx/sgx_repo/ubuntu/intel-sgx-deb.key | sudo apt-key add -
sudo add-apt-repository "deb https://download.01.org/intel-sgx/sgx_repo/ubuntu noble main"
sudo apt-get update
sudo apt-get install -y sgx-aesm-service libsgx-dcap-default-qpl
sudo ln -s /usr/lib/x86_64-linux-gnu/libdcap_quoteprov.so.1 /usr/lib/x86_64-linux-gnu/libdcap_quoteprov.so
  1. Configure the quote provider to use Intel PCS for attestations
sudo cp /etc/sgx_default_qcnl.conf /etc/sgx_default_qcnl.conf.bkp
sudo sed -s -i 's/localhost:8081/api.trustedservices.intel.com/' /etc/sgx_default_qcnl.conf
  1. Install the Rust programming language
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
rustup toolchain default stable
  1. Install sgxs tools from fortanix
cargo install sgxs-tools
  1. Add the current user to the sgx group
sudo groupadd sgx_prv
sudo groupadd sgx
sudo usermod -aG sgx_prv $USER
sudo usermod -aG sgx $USER

Logout and re-login to ensure the group is fully added to your user.

  1. Verify sgx installation

From the sgxs-tools crate we installed, we can verify if the machine has been provisioned correctly by running:

sgx-detect

You should see an output similar to the one below:

Detecting SGX, this may take a minute...
✔  SGX instruction set
  ✔  CPU support
  ✔  CPU configuration
  ✔  Enclave attributes
  ✔  Enclave Page Cache
  SGX features
    ✔  SGX2  ✔  EXINFO  ✔  ENCLV  ✔  OVERSUB  ✔  KSS
    Total EPC size: 378.0MiB
✔  Flexible launch control
  ✔  CPU support
  ？ CPU configuration
  ✔  Able to launch production mode enclave
✔  SGX system software
  ✔  SGX kernel device (/dev/sgx_enclave)
  ✔  libsgx_enclave_common
  ✔  AESM service
  ✔  Able to launch enclaves
    ✔  Debug mode
    ✔  Production mode
    ✔  Production mode (Intel whitelisted)
 
You're all set to start running SGX programs!

Docker compose

Create a directory for your node with the following structure:

nomad/
├─ data/
│  └─ config.toml
└─ docker-compose.yml

For example:

# create both directories
mkdir -p nomad/data
# create the configuration files
touch nomad/docker-compose.yml
touch nomad/data/config.toml

Example docker-compose.yml

The following docker compose file runs the nomad node with the required sgx devices connected, configuration and state mounted, node auto-restart, and a healthcheck script:

services:
  nomad:
    container_name: nomad
    hostname: nomad
    image: "..." # Unreleased for now
    command: ["run", "-v"]
    devices:
      # Allow access to sgx devices
      - /dev/sgx_enclave:/dev/sgx_enclave
      - /dev/sgx_provision:/dev/sgx_provision
    volumes:
      # Mount aesmd dir for socket access
      - /var/run/aesmd:/var/run/aesmd
      # Mount quote provider configuration
      - ./sgx_default_qcnl.conf:/etc/sgx_default_qcnl.conf
      # Mount configuration and state directory
      - ./data:/root/.config/nomad
    environment:
      # Enable sgx logs
      - SGX_DEBUG=1
      # Enable rust backtraces
      - RUST_BACKTRACE=0
    ports:
      # Forward networking ports from configuration
      - "8000:8000"
      - "9000:9000"
    healthcheck:
      # Healthcheck endpoint integration
      test: ["CMD", "curl", "-f", "http://localhost"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 10s
    restart: unless-stopped

Telemetry

Telemetry can be exported by adding standard OTEL_EXPORTER environment variables to the compose file:

For example, using Signoz Cloud:

services:
  nomad:
    ...
    environment:
      - OTEL_EXPORTER_OTLP_ENDPOINT=https://ingest.us.signoz.cloud
      - OTEL_EXPORTER_OTLP_HEADERS=signoz-ingestion-key=<key>

You may also create a .env file next to the docker-compose.yml containing the variables, and expose it to the node:

services:
  nomad:
    ...
    env: .env

Example config.toml

Next to your docker compose file, the data/ directory is exposed to the container for configuration and enclave state.

Create a configuration for the node in data/config.toml:

[enclave]
# Mainnet ethereum bootstrap node
nodes = ["15.235.50.174:8000"]
# Enclave binary configuration, do not touch
enclave_path = "nomad-enclave-0.1.0.sgxs"
signature_path = "nomad-enclave-0.1.0.sig"
signal_log_path = "signal.log"
 
[eth]
# Minimum amount of eth each account should have when swapping tokens
min_eth = 0.0002
# Attested eth rpc url
geth_rpc = "..."
# Attested buildernet rpc url
builder_rpc = "..."
# Mirage platform compliance signers
whitelist_signers = ["9070a3c16a130e1db8f27414f3c6081c99798c7c856670ae49ac2f10b3faa4a4"]
 
[eth.uniswap]
# Enable swapping configured tokens for ETH to maintain EOA transaction funds
enabled = false
# Number of active eoa accounts to manage funds inside
num_accounts = 2
# Mainnet ethereum uniswap router - do not touch unless deploying on an alternate network
router = "0x7a250d5630B4cF539739dF2C5dAcb4c659F2488D"
# Minimum amount of eth to swap for at a time
# If the node needs to swap more than this, a multiple is used to get to at least min_eth
target_eth_amount = 0.005
max_slippage_percent = 5
swap_deadline = "20m"
check_interval = "5m"
 
[eth.token.USDT]
# Mainnet USDT token address
address = "0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48"
# Minimum total balance to reserve when swapping (in human-readable units)
min_balance = 5000.0
# Enable swapping from this token
swap = true
 
[p2p]
# List of peer multiaddrs to bootstrap from
bootstrap = ["/ip4/15.235.50.174/tcp/9000"]
# Interval for searching for more network peers
bootstrap_interval = "5m"
# Port to listen on for the node's libp2p tcp server.
# Must be the same as the port forwarded in the docker container.
tcp = 9000
 
[api]
# Port to listen on for the node's rest api server.
# Must be the same as the port forwared in the docker container.
port = 8000
 
[otlp]
# Opentelemetry sources, exporter configured using standard opentelemetry
# environment variables
logs = false
traces = false
metrics = false

Funding the node

Node runners can onboard new clean funds by sending them to the address printed by the deposit subcommand:

docker compose run nomad deposit

The node will move the funds into encumbered accounts and uses them to execute user signals. Direct CEX transfers are best to ensure the smoothest onboarding process.

Starting and updating the node

Once the configuration has been created and funds have been deposited, the node can be started and managed using normal docker compose commands:

docker compose up -d --pull always

It is required to re-run this command with --pull always periodically, to ensure the node software is up-to-date. Failure to do so may lead to missing out on signals and rewards.

Stopping and withdrawing from the node

At any point the node operator can request funds to be withdrawn from the node's accounts. Simply halt the node process, and run one of the the following commands:

# Stop the node first
docker compose down
 
# For all funds
docker compose run nomad withdraw --all <RECIPIENT>
 
# For a specific amount of funds.
# Amount MUST include token decimals, for example 5000 USDC with 6 decimals:
docker compose run nomad withdraw --contract <TOKEN_ADDRESS> --amount 5000000000 <RECIPIENT>

The node will begin the withdraw process inside the enclave, sending the platform any outstanding fees, producing Mirage signals, and broadcasting them to other nodes for final transfers.