Your chosen agent connected to a local model in one command
Verify the OS version and GPU are visible before installing anything.
cat /etc/os-release | head -n 2
nvidia-smi
Expected output should show a supported Linux OS for your hardware platform and a detected GPU.
Install Ollama or confirm your install supports ollama launch.
curl -fsSL https://ollama.com/install.sh | sh
ollama --version
If Ollama is already installed, just verify the version:
ollama --version
Expected output should show a current Ollama release.
Download the Qwen3.6 model weights to your hardware platform.
ollama pull qwen3.6:35b-a3b-mtp-q4_K_M
The default is the MTP Q4_K_M variant recommended in the Supported hardware platforms matrix. Optional higher-memory variants:
ollama pull qwen3.6:35b-a3b-q8_0 # Higher-quality 8-bit quant (~39GB)
ollama pull qwen3.6:35b-a3b-bf16 # Full precision (~71GB)
Expected output should show qwen3.6:35b-a3b-mtp-q4_K_M (and any optional variants) in ollama list.
Run a quick prompt to confirm the model loads.
ollama run qwen3.6:35b-a3b-mtp-q4_K_M
Try a prompt like:
Write a short README checklist for a Python project.
Expected output should show the model responding in the terminal. When you are done, type /bye or press Ctrl+D to exit the interactive session before continuing.
Install Claude Code, then use Ollama's built-in launch method to start Claude Code against your local model. No environment variables or config files are required.
curl -fsSL https://claude.ai/install.sh | bash
claude --version
If Claude Code is already installed, just verify the version:
claude --version
ollama launch claude --model qwen3.6:35b-a3b-mtp-q4_K_M
Expected output should show Claude Code starting and using the local Qwen3.6 model. Qwen3.6 ships with a 256K context window by default; adjust context length through Ollama's settings if you need to tune it further.
Create a tiny repo and let Claude Code implement a function and tests.
mkdir -p ~/cli-agent-demo
cd ~/cli-agent-demo
printf 'def add(a, b):\n """Return the sum of a and b."""\n pass\n' > math_utils.py
printf 'import math_utils\n\n\ndef test_add():\n assert math_utils.add(1, 2) == 3\n' > test_math_utils.py
If you do not already have pytest installed:
python3 -m venv .venv
source .venv/bin/activate
python3 -m pip install -U pytest
In Claude Code:
Please implement add() in math_utils.py and make sure the test passes.
Run the test:
python3 -m pytest -q
Expected output should show the test passing. When you are done, run deactivate to exit the virtual environment.
Remove the model and stop services if you no longer need them. Cleanup is optional.
To stop the service:
sudo systemctl stop ollama
WARNING
This will delete the downloaded model files.
ollama rm qwen3.6:35b-a3b-mtp-q4_K_M
q8_0 or bf16 variants for higher-quality, higher-memory tradeoffs if your hardware platform has enough memory