Run Hermes Agent with a Local LLM | DGX Spark

Run Hermes Agent with a Local LLM

30 MIN

Chat from the terminal against local vLLM with the self-improving Nous Research agent (Telegram optional)

Overview Instructions Agent-ready Models Troubleshooting

Agent-ready models for Hermes

Hermes expects an OpenAI-compatible local endpoint. This playbook uses vLLM and the agent-ready model recipe validated for the supported hardware platform below.

Serve the model with the Serve LLMs with vLLM playbook (Agent-ready Models tab), then return here to install and configure Hermes.

Recommended models by hardware platform

Hardware platform	Recommended agent-ready model	Serve with
DGX Spark	Agent-ready Qwen3.6-35B-A3B (NVFP4) — `nvidia/Qwen3.6-35B-A3B-NVFP4`	vLLM playbook — Agent-ready Models

Only platforms listed in the Supported hardware platforms table (Overview) are listed above.

After the model is serving

Confirm the OpenAI-compatible API on port 8000:

Shell

curl -sS http://localhost:8000/v1/models

You should see a JSON "data" array that includes your served model handle.

Continue with the Instructions tab to install Hermes and point it at http://localhost:8000/v1.

Run Hermes Agent with a Local LLM

Agent-ready models for Hermes

Recommended models by hardware platform

After the model is serving

Resources