Run OpenClaw with a Local LLM
Install a local-first AI agent and connect it to a private OpenAI-compatible model endpoint
Agent-ready models
Agent-ready models are tuned for agentic workloads — tool calling, reasoning traces, and long multi-turn sessions. Use this tab to pick a recommended local model for your hardware platform, then return to the Instructions tab to connect OpenClaw to the OpenAI-compatible endpoint.
Recommendations by hardware platform
| Hardware platform | Recommended model | Example handle / tag | Inference path |
|---|---|---|---|
| DGX Spark | Agent-ready Qwen3.6-35B-A3B (NVFP4) | nvidia/Qwen3.6-35B-A3B-NVFP4 | vLLM — see Serve LLMs with vLLM → Agent-ready Models |
| RTX (24GB+ VRAM) | Qwen3.6 27B | qwen/qwen3.6-27b (LM Studio) or qwen3.6:27b (Ollama) | LM Studio, Ollama, or vLLM |
| RTX (12–16GB VRAM) | Qwen 3.5 9B / Gemma 4 12B | backend-specific tags | LM Studio or Ollama |
| RTX (6–8GB VRAM) | Qwen 3.5 4B | backend-specific tags | LM Studio or Ollama |
Only platforms listed in the Supported hardware platforms table (Overview) are listed above.
Context window guidance
For OpenClaw, set the local server context window to at least 32K tokens. If your hardware platform has additional memory headroom, 64K or higher is recommended so multi-turn agent sessions and skills have room to work.
Large-memory Linux hardware platforms — vLLM agent-ready path
On large-memory Linux hardware platforms, use the agent-ready nvidia/Qwen3.6-35B-A3B-NVFP4 recipe from the Serve LLMs with vLLM playbook’s Agent-ready Models tab. That path serves an OpenAI-compatible API (typically at http://localhost:8000/v1) suitable for OpenClaw.
- Follow the Serve LLMs with vLLM playbook’s Agent-ready Models tab for the launch command for your hardware platform.
- Verify:
curl http://localhost:8000/v1/models
- In OpenClaw, set provider
baseUrltohttp://localhost:8000/v1and setid/nametonvidia/Qwen3.6-35B-A3B-NVFP4. MatchcontextWindowto the server--max-model-len(the agent-ready recipe uses a large context suitable for multi-turn agent sessions).
Discrete-GPU hardware platforms — LM Studio / Ollama / vLLM
On discrete-GPU hardware platforms, choose a model that fits available VRAM (table above), then serve it with your preferred backend:
- LM Studio — easy GUI path; MTP may be enabled by default on supported models.
- Ollama — simple CLI path.
- vLLM — maximum configurability on Linux; use when you want OpenAI-compatible HTTP serving and custom flags.
Example LM Studio download + serve (24GB+ recommendation):
lms get qwen/qwen3.6-27b
lms load qwen/qwen3.6-27b --context-length 65536
lms server start
Example Ollama pull + context setting:
ollama pull qwen3.6:27b
ollama run qwen3.6:27b
# then: /set parameter num_ctx 65536
Use the exact model handle or tag as the OpenClaw Model ID.