Run Hermes Agent with a Local LLM

Basic idea

Hermes Agent is a self-improving AI agent built by Nous Research. It runs as a terminal TUI on your machine and, through a built-in gateway, can also be reached from messaging platforms like Telegram, Discord, and Slack. It creates skills from experience, improves them during use, persists memory across sessions, and can run scheduled tasks via its built-in cron.

Running Hermes and its LLM fully on your hardware platform keeps conversations and data private and avoids ongoing cloud API costs. You can serve a large local model for stronger reasoning quality and chat from the terminal. An optional messaging gateway (for example Telegram) can reach the same agent from your phone while inference stays local.

What you'll accomplish

Install Hermes on your hardware platform and connect it to a local LLM served by vLLM (the agent-ready nvidia/Qwen3.6-35B-A3B-NVFP4 recipe). Chat with the agent from the terminal. Optionally set up a Telegram bot later if you want messaging access.

• Serve a local model with vLLM
• Install Hermes and configure it against the local vLLM endpoint
• Chat from the Hermes terminal (TUI)
• Optionally set up Telegram messaging (see Instructions — optional section at the bottom)
• Resume past sessions, switch models, update, and uninstall using the hermes CLI

Popular use cases

• Personal secretary: With access to your inbox, calendar, and files, OpenClaw can help manage your schedule, draft replies, send reminders, and find meeting slots.
• Proactive project management: Check project status over email or messaging, send status updates, and follow up or send reminders.
• Research agent: Combine web search and your local files to produce reports with personalized context.
• Install helper: Search for apps/libraries, run installations, and debug errors using terminal access (larger models recommended).

What to know before starting

Required:

• Basic use of the Linux terminal and a text editor
• Familiarity with Docker and vLLM, or willingness to follow the Serve LLMs with vLLM playbook first
• Awareness of the security considerations below

Optional:

• A Telegram account if you want to use the messaging gateway

Supported hardware platforms

Use the matrix below to confirm your hardware platform and the default local inference recipe used in this playbook.

Important: security and risks

AI agents that can execute commands and reach external services introduce real risks. Read the upstream guidance, especially the dedicated security topics: Hermes Agent — Security.

Main risks:

1. Data exposure: Personal information or files on your hardware may be leaked through agent actions or optional messaging channels.
2. Unauthorized access: A messaging bot left open to anyone who finds it can be misused; a model endpoint exposed beyond localhost can be abused.

You cannot eliminate all risk; proceed at your own risk. Recommended security measures:

• If you enable Telegram (optional), restrict the bot by entering one or more numeric Telegram user IDs at the "Allowed user IDs" prompt. Leaving this blank allows anyone who finds the bot to use it.
• Keep the vLLM endpoint bound to the hardware platform; do not forward http://<host-ip>:8000 to your LAN or the public internet without strong authentication.
• Run Hermes on hardware dedicated to this purpose where possible, and only place files on it that the agent is allowed to access.
• Monitor activity: Periodically review Hermes session history and, if you installed a gateway, its service logs (sudo journalctl -u <hermes-gateway-unit> -e).

Prerequisites

Hardware requirements

• Supported hardware platform — see Supported hardware platforms matrix above
• Sufficient memory for the agent-ready model (see Agent-ready Models tab)

Software requirements

• Linux with terminal (SSH or local) access
• curl and git installed (verified in Step 1 of the instructions)
• Interactive terminal access for the setup wizard and any sudo password prompts. Non-interactive SSH is supported with the config-command fallback in the Instructions tab.
• Docker with the NVIDIA Container Toolkit, plus a HuggingFace token to download the model
• (Optional but recommended) A supported messaging account (e.g. Telegram) to access the agent remotely

Time & risk

• Estimated time: 30 MIN for install and first-time terminal setup; model download time depends on size and network speed. Optional Telegram setup adds time.
• Risk level: Medium — the agent can execute commands and persist skills. Risk increases if you enable an optional messaging gateway without restricting allowed user IDs, or expose the local model endpoint beyond localhost. Always follow the security measures above.
• Rollback: Run hermes uninstall (with sudo if you installed a gateway as a system service) to remove Hermes, any gateway service, and the shell-profile entry. The data directory ~/.hermes may still be present afterward; remove it manually if you want a full reset (see Cleanup in Instructions and the Troubleshooting tab). Stop the vLLM container separately (docker rm / docker rmi) if desired.
• Last Updated: 07/27/2026
  • Telegram messaging moved to an optional section at the bottom of Instructions; core path is terminal-only