--- title: "Run Hermes Agent with a Local LLM" publisher: "nvidia" type: "playbook" updated: "2026-08-04T16:15:26.715Z" description: "Chat from the terminal against local vLLM with the self-improving Nous Research agent (Telegram optional)" canonical: "https://build.nvidia.com/playbooks/hermes-agent.md" --- # Basic idea [Hermes Agent](https://github.com/NousResearch/hermes-agent) is a **self-improving** AI agent built by [Nous Research](https://nousresearch.com). It runs as a terminal TUI on your machine and, through a built-in gateway, can also be reached from messaging platforms like Telegram, Discord, and Slack. It creates skills from experience, improves them during use, persists memory across sessions, and can run scheduled tasks via its built-in cron. Running Hermes and its LLM **fully on your hardware platform** keeps conversations and data private and avoids ongoing cloud API costs. You can serve a large local model for stronger reasoning quality and chat from the terminal. An optional messaging gateway (for example Telegram) can reach the same agent from your phone while inference stays local. # What you'll accomplish Install Hermes on your hardware platform and connect it to a local LLM served by **vLLM** (the agent-ready `nvidia/Qwen3.6-35B-A3B-NVFP4` recipe). Chat with the agent from the terminal. Optionally set up a Telegram bot later if you want messaging access. - Serve a local model with vLLM - Install Hermes and configure it against the local vLLM endpoint - Chat from the Hermes terminal (TUI) - Optionally set up Telegram messaging (see Instructions — optional section at the bottom) - Resume past sessions, switch models, update, and uninstall using the `hermes` CLI # Popular use cases - **Personal secretary**: With access to your inbox, calendar, and files, OpenClaw can help manage your schedule, draft replies, send reminders, and find meeting slots. - **Proactive project management**: Check project status over email or messaging, send status updates, and follow up or send reminders. - **Research agent**: Combine web search and your local files to produce reports with personalized context. - **Install helper**: Search for apps/libraries, run installations, and debug errors using terminal access (larger models recommended). # What to know before starting **Required:** - Basic use of the Linux terminal and a text editor - Familiarity with Docker and vLLM, or willingness to follow the [Serve LLMs with vLLM](https://build.nvidia.com/playbooks/vllm) playbook first - Awareness of the security considerations below **Optional:** - A Telegram account if you want to use the messaging gateway # Supported hardware platforms Use the matrix below to confirm your hardware platform and the default local inference recipe used in this playbook. | Hardware platform | OS | Memory | Multi-node capable hardware | | :---- | :---- | :---- | :---- | | **DGX Spark** | DGX OS (Linux) | 128 GB Unified Memory | — | > [!NOTE] Only platforms listed in the Supported hardware platforms table above are covered by this playbook. # Important: security and risks AI agents that can execute commands and reach external services introduce real risks. Read the upstream guidance, especially the dedicated security topics: [Hermes Agent — Security](https://hermes-agent.nousresearch.com/docs/user-guide/security). Main risks: 1. **Data exposure**: Personal information or files on your hardware may be leaked through agent actions or optional messaging channels. 2. **Unauthorized access**: A messaging bot left open to anyone who finds it can be misused; a model endpoint exposed beyond `localhost` can be abused. You cannot eliminate all risk; proceed at your own risk. **Recommended security measures:** - If you enable Telegram (optional), **restrict the bot** by entering one or more numeric Telegram user IDs at the *"Allowed user IDs"* prompt. Leaving this blank allows anyone who finds the bot to use it. - Keep the vLLM endpoint bound to the hardware platform; do not forward `http://:8000` to your LAN or the public internet without strong authentication. - Run Hermes on hardware dedicated to this purpose where possible, and only place files on it that the agent is allowed to access. - **Monitor activity**: Periodically review Hermes session history and, if you installed a gateway, its service logs (`sudo journalctl -u -e`). # Prerequisites **Hardware requirements** - Supported hardware platform — see Supported hardware platforms matrix above - Sufficient memory for the agent-ready model (see **Agent-ready Models** tab) **Software requirements** - Linux with terminal (SSH or local) access - `curl` and `git` installed (verified in Step 1 of the instructions) - Interactive terminal access for the setup wizard and any `sudo` password prompts. Non-interactive SSH is supported with the config-command fallback in the Instructions tab. - Docker with the NVIDIA Container Toolkit, plus a HuggingFace token to download the model - (Optional but recommended) A supported messaging account (e.g. Telegram) to access the agent remotely # Time & risk - **Estimated time:** 30 MIN for install and first-time terminal setup; model download time depends on size and network speed. Optional Telegram setup adds time. - **Risk level:** **Medium** — the agent can execute commands and persist skills. Risk increases if you enable an optional messaging gateway without restricting allowed user IDs, or expose the local model endpoint beyond `localhost`. Always follow the security measures above. - **Rollback:** Run `hermes uninstall` (with `sudo` if you installed a gateway as a system service) to remove Hermes, any gateway service, and the shell-profile entry. The data directory `~/.hermes` may still be present afterward; remove it manually if you want a full reset (see Cleanup in Instructions and the Troubleshooting tab). Stop the vLLM container separately (`docker rm` / `docker rmi`) if desired. - **Last Updated:** 07/27/2026 - Telegram messaging moved to an optional section at the bottom of Instructions; core path is terminal-only ## More - [Instructions](/playbooks/hermes-agent/instructions.md) - [Agent-ready Models](/playbooks/hermes-agent/agent-ready-models.md) - [Troubleshooting](/playbooks/hermes-agent/troubleshooting.md)