---
title: "Serve Models with llama.cpp — Troubleshooting"
canonical: "https://build.nvidia.com/playbooks/llama-cpp/troubleshooting.md"
---

| Symptom | Hardware platform | Cause | Fix |
|---------|-------------------|-------|-----|
| `cmake` fails with "CUDA not found" | All hardware platforms | CUDA toolkit not in PATH | Run `export PATH=/usr/local/cuda/bin:$PATH` and re-run CMake from a clean build directory |
| Build errors mentioning wrong GPU arch | All hardware platforms | CMake `CMAKE_CUDA_ARCHITECTURES` does not match the GPU | Use `-DCMAKE_CUDA_ARCHITECTURES=121a-real` as in the Instructions tab |
| GGUF download fails or stalls | All hardware platforms | Network or Hugging Face availability | Re-run the `-hf` launch from Step 4; it resumes partial files |
| "CUDA out of memory" when starting `llama-server` | All hardware platforms | Model too large for current context or available memory | Lower `--ctx-size` (e.g. 4096) or use a smaller quantization from the same repo |
| Server runs but latency is high | All hardware platforms | Layers not on GPU | Check `nvidia-smi` during a request to confirm the model is on the GPU; if not, set `--n-gpu-layers` explicitly as described in Step 4 |
| `curl: (7) Failed to connect` on port 30000 | All hardware platforms | No listener yet, wrong host, or crash | Wait for `server is listening`; run `curl` on the same host as `llama-server` (or use `<HARDWARE_IP>`); run `ss -tln` and confirm `:30000`; read server stderr for OOM or a bad `--model` path |
| Chat API errors or empty replies | All hardware platforms | Wrong `--model` path or incompatible GGUF | Verify the path to the `.gguf` file; update llama.cpp if the GGUF requires a newer format |
| Memory pressure within capacity | DGX Spark | UMA buffer cache not released | See UMA note below |

> [!NOTE]
> **Unified memory (UMA).** On hardware platforms with unified memory, GPU and CPU share memory dynamically. Some applications have not yet been updated for UMA, so you may hit memory issues even within capacity. If that happens, manually flush the buffer cache (use with care on shared systems):
> ```bash
> sudo sh -c 'sync; echo 3 > /proc/sys/vm/drop_caches'
> ```

For latest known issues, see the documentation linked under **Resources** for your hardware platform.