---
title: "Run Hermes Agent with a Local LLM — Instructions"
canonical: "https://build.nvidia.com/spark/hermes-agent/instructions.md"
---

> [!NOTE] These instructions target **Linux** on a supported hardware platform. Keep a separate terminal running vLLM while you install and use Hermes. Telegram messaging is **optional** — complete Steps 1–8 for terminal use first; only then follow the optional Telegram section at the bottom if you want a messaging gateway.

# Step 1. Verify your environment

Before installing Hermes, confirm that your hardware platform is running Linux, has network access, and exposes the basic command-line tools used during install.

```shell
uname -a
curl --version
git --version
```

**What to look for:** A healthy system typically shows **Linux** in the `uname -a` output. Confirm that `curl --version` and `git --version` print version lines without errors.

# Step 2. Serve a model with vLLM

Hermes will be configured against a local, OpenAI-compatible endpoint, so a model server must be running before you launch the Hermes installer. This playbook uses **vLLM** with the agent-ready recipe for your hardware platform — see the **Agent-ready Models** tab.

Follow the [Serve LLMs with vLLM](https://build.nvidia.com/spark/vllm) playbook’s Agent-ready Models guidance to launch the server in a **separate terminal** so it can run alongside Hermes. It serves an OpenAI-compatible API at `http://localhost:8000/v1`.

Once the server reports `Application startup complete`, verify the API on **8000** in another terminal. A healthy server returns **JSON** with a top-level **`"data"`** array listing the served model:

```shell
curl -sS http://localhost:8000/v1/models
```

You should see your served model handle (for example `nvidia/Qwen3.6-35B-A3B-NVFP4`) in the returned list.

> [!NOTE] Keep the vLLM endpoint bound to the hardware platform only. The container publishes port `8000`; do not forward `http://<host-ip>:8000` to your LAN or the public internet without strong authentication.

# Step 3. Install Hermes

The steps below were verified against **Hermes Agent v0.18.0 (2026.7.1) · upstream `676236bb`**. Newer installer versions may word prompts differently or reorder them; follow the closest matching prompt.

Run the installer from an **interactive terminal** on the hardware platform. If you are connected over SSH, use a normal SSH session where you can answer prompts and enter your `sudo` password when requested. If you run the installer from a non-interactive automation shell, Hermes can install but the setup wizard and optional system-package prompts may be skipped; use the **Non-interactive SSH fallback** below in that case.

```shell
curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.sh | bash
```

The installer walks you through an interactive setup wizard. This playbook uses the **Blank Slate** path, which keeps the wizard short — basics, model endpoint, and terminal backend. Messaging gateways (including Telegram) are optional and configured later only if you want them. The subsections below follow the wizard's prompt order.

> [!IMPORTANT] **OpenClaw on the same machine (out of scope for this playbook):** If another tool such as **OpenClaw** was installed previously, the Hermes installer may ask whether you want to **import** or **migrate** from it. For the steps in *this* playbook, answer **`n`** (no) so Hermes does not pull in OpenClaw configuration. Mixing migrations can leave gateway or messaging state inconsistent; if you already migrated by mistake, prefer a clean reinstall (see **Start over from scratch** in the Troubleshooting tab) before continuing.

## 3a. Installer basics

1. **"Install ripgrep for faster file search ffmpeg for TTS voice messages? [Y/n]"** — Press **Enter** to accept the default and install both helpers. If `sudo` asks for your password, enter your Linux user password. If you skip this step or run without a terminal, Hermes still works, but file search falls back to slower tools and TTS voice-message support is limited. You can install the helpers later with `sudo apt install -y ripgrep ffmpeg`.  

2. **"How would you like to set up Hermes?"** — Choose **Blank Slate**. It starts with everything off except the bare minimum, still offers the provider prompts used in the rest of this step, and lets you opt in to individual capabilities later. For context on the other options: **Quick Setup** does not offer provider selection — it signs in through [**https://portal.nousresearch.com**](https://portal.nousresearch.com) (registration or login required) instead, so it won't point Hermes at the local model endpoint on your hardware — and **Full setup** walks through every provider, tool, and plugin option — this playbook uses Blank Slate to get users started quickly.

## 3b. Point Hermes at your local model

1. **"Select provider"** — The wizard lists many hosted providers; scroll past them and choose **Custom endpoint (enter URL manually)** so Hermes can be pointed at the model endpoint running on your hardware. (If the machine already has saved endpoints, entries like **Local (localhost:8000)** appear near the bottom of the list — selecting the matching saved entry works too.)  

2. **"API base URL [e.g. https://api.example.com/v1]:"** — *If this prompt appears*, enter the URL of your local model server. For the local vLLM endpoint from Step 2, use `http://localhost:8000/v1`. (Depending on installer version or prior config, this question is sometimes skipped when the endpoint is already inferred—continue with the prompts you do see.)  

3. **"API key [optional]"** — Leave blank and press **Enter**; vLLM does not require a key for a local model.  

4. **Model selection** — The installer lists the models served by your local endpoint (vLLM reports these via `/v1/models`). Select the agent-ready handle from the **Agent-ready Models** tab (for example `nvidia/Qwen3.6-35B-A3B-NVFP4`).  

5. **"Context length in tokens [leave blank for auto-detect]:"** — Press **Enter** to let Hermes auto-detect the context length from the served model (the agent-ready recipe serves `--max-model-len 262144`).  

6. **"Display name [Local (localhost:8000)]"** — Press **Enter** to accept the suggested label, or type a custom name to identify this endpoint in the Hermes UI.

**Non-interactive SSH fallback:** If the installer prints **"Setup wizard skipped (no terminal available)"**, or if you are validating the playbook through non-interactive SSH, configure the local vLLM endpoint with Hermes' config command instead of the prompts above:

```shell
export PATH="$HOME/.local/bin:$PATH"
hermes config set model.provider custom
hermes config set model.base_url http://localhost:8000/v1
hermes config set model.default nvidia/Qwen3.6-35B-A3B-NVFP4
hermes -z "Reply exactly HERMES_OK"
```

The last command should return `HERMES_OK`, confirming that Hermes can call the local vLLM model without opening the TUI.

## 3c. Choose the terminal backend

1. **"Select terminal backend:"** — Choose **Keep current (local)** (or **Local** if no current option is shown). This determines where Hermes executes shell commands when the agent uses its terminal tool; the local backend runs them directly on the hardware platform, which is what this playbook assumes. The other options (Docker, Modal, SSH, Daytona, Singularity/Apptainer) run commands in containers or off-box environments and are out of scope here.

## 3d. Finish the wizard and verify

1. **"Your minimal agent is ready. What next?"** — Choose **Start with everything disabled — finish now (most minimal)**. You can enable capabilities later at any time.  

2. **Reload your shell** to make the `hermes` command available, then verify the command resolves:

```shell
source ~/.bashrc
export PATH="$HOME/.local/bin:$PATH"
which hermes
```

3. **Verify the model wiring.** Run `hermes` to open the TUI, type `hello`, and press **Enter**; the agent should respond, confirming that the model endpoint and Hermes are wired up correctly. When you're done, type `/exit` to leave the chat and return to your shell. On exit, Hermes prints the exact command needed to resume this conversation later — `hermes --resume <sessionId>`. Save it if you want to pick up where you left off.

You now have a working Hermes install for terminal use. Steps 4–8 below are optional operational tasks. Messaging (Telegram) is covered only in the **optional** section at the end of this page.

# Step 4. Switch to a different model

You configured an initial model during the Hermes install. To switch to a different one later, restart vLLM serving the new model handle, then re-point Hermes at the same local endpoint.

1. Stop the current vLLM container (Ctrl+C in its terminal) and relaunch it with the new model handle. Use the launch settings from the [Serve LLMs with vLLM](https://build.nvidia.com/spark/vllm) playbook (Agent-ready Models), swapping the model handle and any flags appropriate for the new model.  

2. Launch the Hermes model picker:

```shell
hermes model
```

3. At the **"Select Provider"** prompt, choose **Custom endpoint (enter URL manually)**.  

4. **If you see the “API base URL” prompt**, enter the same local vLLM endpoint as before:

```
http://localhost:8000/v1
```

5. When Hermes lists the models served by the endpoint, choose the one you just started serving. Hermes will use it for subsequent sessions.

If you are in a non-interactive SSH session, switch models with config commands instead:

```shell
hermes config set model.provider custom
hermes config set model.base_url http://localhost:8000/v1
hermes config set model.default <new-model-handle>
hermes -z "Reply exactly MODEL_OK"
```

# Step 5. Configure tools

The Blank Slate install from Step 3 starts with the agent's tools disabled. To enable more tools or modify the existing tool configuration, run:

```shell
hermes tools
```

This opens the same multi-select tool list as the setup wizard (web search, browser automation, terminal, file operations, code execution, and more). Toggle entries with **SPACE** and press **ENTER** to confirm.

# Step 6. Resume a previous Hermes session

To pick up a past conversation, launch Hermes with the `--resume` flag and the session ID printed when you exited that chat:

```shell
hermes --resume <sessionId>
```

The TUI will reopen with the prior conversation history restored, ready for follow-up prompts.

# Step 7. Update Hermes

To upgrade an existing Hermes installation to the latest release, run:

```shell
hermes update
```

The command pulls the latest Hermes version, applies any required dependency changes, and restarts the gateway service if one is installed so the new version takes effect.

# Step 8. Cleanup

> [!WARNING] This removes the Hermes installation and any gateway service you configured. By default, `~/.hermes/` (configuration, conversation history, and skills) is preserved unless you opt into a full uninstall at the on-screen prompt.

Run cleanup from an **interactive terminal**. The uninstaller may refuse non-interactive subprocesses and still asks you to choose whether to keep data or perform a full uninstall. For a full wipe, choose **Full uninstall** and type **`yes`** at the confirmation prompt.

If you installed a messaging gateway as a **System service** (optional Telegram section below), run the uninstall with `sudo` so it has permission to remove the system-scope systemd unit. If `sudo hermes uninstall` fails with **command not found**, it is because `sudo` does not inherit your user `PATH`; invoke the binary by its full path instead:

```shell
export PATH="$HOME/.local/bin:$PATH"
HERMES_BIN="$(command -v hermes || printf '%s\n' "$HOME/.local/bin/hermes")"
sudo "$HERMES_BIN" uninstall
```

If you never installed a system gateway service, `hermes uninstall` as your normal user is enough.

Follow the on-screen prompts to confirm removal. The uninstaller typically:

- Stops and removes the systemd gateway service (if present).  
- Removes the `hermes` wrapper script and the PATH entries added to your shell profile.  
- Deletes the Hermes application directory.

**Data directory:** The **`~/.hermes`** directory (configuration, sessions, skills) is **not always removed** by `uninstall`, depending on the options you choose at prompts. After uninstall, check whether it still exists:

```shell
ls -la ~/.hermes
```

If you intend a **full** removal, delete it manually (this is irreversible):

```shell
rm -rf ~/.hermes
```

# Step 9. Tips & Tricks 

1. **Inspect the agent's reasoning for observability.** Inside the TUI, run `/reasoning show` to surface the model's intermediate reasoning alongside its responses. This is especially useful for following the agent's progress on multi-step or complex problems and for debugging unexpected answers.  
2. **Try a multi-step task to trigger skill creation.** For example, ask the agent how to set up email — Hermes will walk through the configuration with you and, on completing a non-trivial task like this, may autonomously persist a reusable skill so the next email-related request is faster.  
3. **Configure scheduled automations via the built-in cron**. For example, ask Hermes to check the price of a product online once a day. If you also set up the optional Telegram gateway below, Hermes can deliver each result through that channel.

---

# Optional — Set up messaging

Skip this entire section if you only need the Hermes terminal (TUI) or Desktop app. Setting up a messaging app requires an account. In this guide, we will walk through setting up Telegram as an example. Telegram requires outbound HTTPS to Telegram’s cloud API and is not required for local chat.

## O1. Verify outbound HTTPS to Telegram

> [!NOTE] On some corporate or lab networks, **outbound HTTPS to `api.telegram.org` is blocked**, which produces a working local install but a **bot that never responds**. Before gateway setup, run this check from your PC

```shell
curl -sS --connect-timeout 10 -o /dev/null -w "HTTP %{http_code}\n" https://api.telegram.org/
```

You should see an **HTTP status line** such as **`HTTP 404`**, **`HTTP 200`**, or **`HTTP 302`**. The important part is that the request **completes over TLS** without hanging. **Timeouts**, **“Could not resolve host”**, or **connection refused** mean the gateway will not reach Telegram from this network—try a path that allows that traffic (for example a personal hotspot) or ask your network administrator to allow **HTTPS to `api.telegram.org`**.

## O2. Configure the Telegram gateway

The Blank Slate install skips messaging setup. Configure Telegram with:

```shell
hermes gateway setup
```

1. **"Select platforms to configure:"** — The command shows a multi-select list of messaging platforms. Navigate to **Telegram**, press **SPACE** to select it, then press **ENTER** to confirm. The remaining steps use Telegram as the example; the same flow applies to other supported gateways.  

> [!TIP] **If Telegram questions are skipped:** Some users see **“Setup complete”** or **“Messaging Platforms (Gateway) configuration complete!”** immediately after choosing Telegram, without token or user-ID prompts. That usually means a prior partial Telegram state exists. Re-run **`hermes gateway setup`** and select Telegram again to supply the bot token and allowed user IDs. (If the CLI suggests `hermes setup gateway` but that flow still skips prompts, use **`hermes gateway setup`**. If it keeps skipping, see **Start over from scratch** in the Troubleshooting tab.) Follow the printed **`sudo`** lines to register the gateway service (if a printed `sudo hermes …` command fails with `command not found`, see the Troubleshooting tab).  

2. **"Telegram bot token:"** — Open Telegram and start a chat with [@BotFather](https://t.me/BotFather), follow its guided flow to create a new bot, then paste the token BotFather returns into this prompt. **Tip:** Installing [Telegram Desktop](https://desktop.telegram.org/) on the same machine as your SSH session lets you **copy the token from Telegram and paste into the terminal** without retyping it from your phone. The terminal will not echo any characters as the token is pasted — this is expected. Press **Enter** to submit; the installer should respond with `Telegram token saved`.  

3. **"Allowed user IDs (comma-separated, leave empty for open access):"** — To restrict the bot to specific Telegram accounts, follow the on-screen instructions to look up your numeric Telegram user ID, then enter one or more IDs separated by commas. Leaving this field blank allows anyone who can reach the bot to use it, which is generally not recommended.  

4. **"Use your user ID (<your-id>) as the home channel? [Y/n]:"** — Press **Enter** to accept. This designates your own Telegram account as the default channel Hermes will use for proactive messages and scheduled deliveries.  

5. **"Install the gateway as a systemd service? (runs in background, starts on boot) [Y/n]:"** — Press **Enter** to accept. The gateway will run as a background service.  

6. **"Choose how the gateway should run in the background:"** — Choose **System service** if you want Hermes to start at boot without requiring an interactive login. The service will still run under your user account so it can read your Hermes configuration; only installation requires `sudo`. If this prompt doesn't appear or you need to (re)install the service manually, run `sudo "$(which hermes)" gateway install --system --run-as-user "$USER"`.  

7. **Verify the gateway.** After configuration, confirm the gateway unit is active and recent logs look healthy (replace `<hermes-gateway-unit>` with the **exact** `*.service` name the installer printed—often something containing `hermes` and `gateway`):

```shell
systemctl list-units --type=service --all | grep -i hermes
systemctl --user list-units --type=service --all | grep -i hermes
sudo systemctl status <hermes-gateway-unit>
sudo journalctl -u <hermes-gateway-unit> -e --no-pager -n 50
```

If `systemctl status` or `systemctl --user status` shows **active (running)** and logs are not repeating connection errors to Telegram, the service side is in good shape. If logs show TLS timeouts or “connection refused” to Telegram hosts, re-run the **outbound HTTPS** check in **O1** above.

8. **Talk to Hermes from Telegram.** The gateway is now running as a background service, so you can reach Hermes from any Telegram client without a terminal session:  

- Open Telegram (mobile or desktop) and search for your bot by the username you assigned through @BotFather.  
- Open the chat with the bot and tap **Start** (or send `/start`) on first contact.  
- Send the message **`hello`**. Hermes will reply through the bot, confirming the gateway is wired to your hardware and the underlying model.

> 

> [!NOTE] After **`/start`**, Telegram may show a generic **“Unknown command”**-style message from the bot. That can be normal for bots that only implement free-form chat. **Ignore that message and send `hello` anyway**—Hermes should respond to normal text once the gateway and model are healthy.

From here you can send any prompt you would normally type in the TUI — Hermes will run on your hardware platform and stream the response back to Telegram.