Run Hermes Agent with a Local LLM
30 MIN
Chat from the terminal against local vLLM with the self-improving Nous Research agent (Telegram optional)
Agent-ready models for Hermes
Hermes expects an OpenAI-compatible local endpoint. This playbook uses vLLM and the agent-ready model recipe validated for the supported hardware platform below.
Serve the model with the Serve LLMs with vLLM playbook (Agent-ready Models tab), then return here to install and configure Hermes.
Recommended models by hardware platform
| Hardware platform | Recommended agent-ready model | Serve with |
|---|---|---|
| DGX Spark | Agent-ready Qwen3.6-35B-A3B (NVFP4) — nvidia/Qwen3.6-35B-A3B-NVFP4 | vLLM playbook — Agent-ready Models |
Only platforms listed in the Supported hardware platforms table (Overview) are listed above.
After the model is serving
- Confirm the OpenAI-compatible API on port 8000:
curl -sS http://localhost:8000/v1/models
You should see a JSON "data" array that includes your served model handle.
- Continue with the Instructions tab to install Hermes and point it at
http://localhost:8000/v1.