Run OpenClaw with a Local LLM

Install with an agent

Use https://build.nvidia.com/playbooks/openclaw.md and complete this playbook on this machine: Run OpenClaw with a Local LLM. Fetch that .md URL directly; the non-.md page is a JavaScript shell. When finished, state clearly whether the playbook succeeded, and give the evidence: what you installed, where, and the output of a command proving it works (a version string, a container listing, or an HTTP status).

Prepare your environment (Windows / WSL only)

If you are on Windows, you can install OpenClaw on native Windows or in Windows Subsystem for Linux (WSL). WSL provides a Linux-style environment if your skills need it; native Windows is often simpler and can connect more easily to Windows apps.

If you choose WSL and it is not already installed:

1. Open PowerShell as Administrator.
2. Install WSL:

wsl --install

3. Verify:

wsl --version

4. Start WSL:

wsl

Install OpenClaw

On your hardware platform, open a terminal and run the official install script:

curl -fsSL https://openclaw.ai/install.sh | bash

On native Windows, follow the install options documented at openclaw.ai if the curl install path does not apply.

After dependencies are downloaded, read the security disclaimer and accept any confirmation if you agree to proceed.

Complete the OpenClaw onboarding

Work through the prompts as follows.

1. How would you like to start? Choose Quick start (recommended).

2. What would you like to create? Choose One agent.

3. Model/auth provider:

   • Recommended for a local model: If your backend is not listed yet (or you will configure it after the server is up), go to the bottom of the list and select Skip for now—you’ll configure the model in a later step.
   • If your inference backend appears in the list (for example Ollama or vLLM), you can select it now and fill in API key (leave blank or use a dummy value), Base URL (default is usually fine), and Model ID (must match the exact handle served by the backend).

After selecting Skip for now, onboarding returns to the shell. Continue with Step 4 to start the model server and Step 5 to configure OpenClaw and start the Gateway. You can add channels, skills, and hooks later.

Serve a local model

OpenClaw connects to a local, OpenAI-compatible endpoint. Pick a backend that fits your hardware platform, then use the matching subsection. Model recommendations are in the Agent-ready Models tab.

Option A — vLLM (recommended on large-memory Linux hardware platforms)

Use when you want an OpenAI-compatible HTTP server and validated agent-ready recipes.

1. In a separate terminal, launch the recommended model for your hardware platform from the Agent-ready Models tab (or follow Serve LLMs with vLLM for container setup).
2. Wait until the server reports startup complete, then verify:

curl http://localhost:8000/v1/models

You should see your model handle in the returned list. The default OpenClaw baseUrl for this path is http://localhost:8000/v1.

Option B — Ollama

curl -fsSL https://ollama.com/install.sh | sh
ollama pull qwen3.6:27b
ollama run qwen3.6:27b

Inside the Ollama session (or via your usual Ollama config), set context high enough for agent use—for example /set parameter num_ctx 65536. Use the exact Ollama model tag as the OpenClaw Model ID.

Configure OpenClaw to use the local server

If you already selected a provider during onboarding and chat works, you can skip to Step 6.

Otherwise, open the OpenClaw config file:

~/.openclaw/openclaw.json

Example with nano:

nano ~/.openclaw/openclaw.json

Add or update the models section so it includes your provider. Example for a vLLM server (no API key required—any non-empty placeholder works):

"models": {
  "mode": "merge",
  "providers": {
    "vllm": {
      "baseUrl": "http://localhost:8000/v1",
      "apiKey": "vllm",
      "api": "openai-responses",
      "models": [
        {
          "id": "nvidia/Qwen3.6-35B-A3B-NVFP4",
          "name": "nvidia/Qwen3.6-35B-A3B-NVFP4",
          "reasoning": true,
          "input": ["text"],
          "cost": {
            "input": 0,
            "output": 0,
            "cacheRead": 0,
            "cacheWrite": 0
          },
          "contextWindow": 262144,
          "maxTokens": 8192
        }
      ]
    }
  }
}

Replace id, name, baseUrl, and contextWindow to match the model and backend you launched. The id and name must match the handle served by the backend.

If the OpenClaw gateway is already running, restart it so it reloads ~/.openclaw/openclaw.json.

Verify the setup

1. In a browser, open the OpenClaw dashboard URL (and use the access token if required).
2. Start a new conversation and send a short message.
3. If you get a reply from the agent, the setup is working.

You can also ask OpenClaw which model it’s using. In the gateway chat UI you can switch models by typing: /model MODEL_NAME.

Optional: add skills and learn more

• Skills add capabilities but also risk; only enable skills you trust (for example, community-vetted ones). To add a skill:

  • Ask OpenClaw to configure a skill, or
  • Use the sidebar in the web UI to enable skills, or
  • Browse Clawhub for community skills.

• For more usage and configuration details, see the OpenClaw documentation.