The Applies to column identifies the relevant path and hardware. "All paths · all hardware platforms" applies throughout this playbook.
| Symptom | Applies to | Cause | Fix |
|---|---|---|---|
| "permission denied" when running docker | Advanced · all hardware platforms | User not in docker group | Run sudo usermod -aG docker $USER && newgrp docker |
| Container fails to start with GPU error | All paths · all hardware platforms | NVIDIA Container Toolkit not configured | Run nvidia-ctk runtime configure --runtime=docker and restart Docker |
| CUDA out of memory | All paths · all hardware platforms | Other models are still running, or the model settings need more memory than is free | For Intermediate, stop other model apps in Sync and check that the launch script matches your device and memory size in Steps 4 and 5. For Advanced, reduce context length or concurrency in the selected recipe, or choose a smaller model or lower-precision checkpoint. |
| Model or draft-model files are missing | Intermediate · all hardware platforms | A download did not finish, the wrong launch script was chosen, or the files were downloaded under another user or cache path | Finish the downloads in Step 4, including the draft model for Qwen3.8 27B or Nemotron 3.5 Bolt. Use the matching script from Step 5 and the same DGX user for downloads and Sync. If you use HF_HOME, check that the script uses the same path as the downloads. |
Could not find a GB300 GPU with nvidia-smi. | Intermediate · DGX Station | The script cannot find the Station's GB300 | Check that Sync is connected to the right Station. Run nvidia-smi on that device and confirm it lists a GB300 before trying again. |
| Port 8000 is in use | Intermediate · Advanced | Another app is using the port on the laptop or DGX device | Stop the other app, or follow the instructions for your path under Use another API port below. |
Laptop cannot reach the API, but /health returns HTTP 200 on the DGX device | Intermediate · Advanced | Port forwarding is not active or uses a different port | For Intermediate, keep Sync connected and the custom app running. Check that the Sync app's Port, the script's PORT, and the laptop URL use the same port. Run the laptop checks in a local terminal as shown in Step 7. For Advanced, check the forwarding route for your client. On Windows, run PowerShell commands in PowerShell rather than WSL unless you have configured forwarding there. |
PowerShell prompts for Uri when you enter a curl command | Intermediate · Advanced | curl is a PowerShell alias for Invoke-WebRequest | Use curl.exe when following a curl command in PowerShell. The playbook's Windows chat-request examples use Invoke-RestMethod. |
Chat request returns The model 'unknown' does not exist | Intermediate · Advanced | The request's model value does not match a served model | Get the model ID from /v1/models and use that exact ID in the chat request. |
torch.OutOfMemoryError on a GPU with about 95 GiB while loading DeepSeek | Advanced (DGX Station) | The recipe's --gpus all let vLLM pick the RTX PRO 6000 instead of the GB300 | Pin the GB300 as in the Advanced tab, Step 3: replace --gpus all with --gpus "device=${GB300_UUID}". |
engine input registration timed out after 3600s during FlashInfer autotune | Advanced (DGX Station) | First-run kernel tuning takes longer than the engine-ready timeout | Raise VLLM_ENGINE_READY_TIMEOUT_S (for example 7200) and mount ~/.cache/vllm and ~/.cache/flashinfer so tuning results persist. |
| Memory pressure within capacity | DGX Spark | UMA buffer cache not released | See UMA note below |
Chat completion returns content: null with finish_reason: length | Intermediate · Advanced | The request's max_tokens was exhausted during reasoning | Increase max_tokens in the request and retry. |
Choose the instructions for the path you are using. Keep the laptop port, DGX host port, and container port distinct; the correct change depends on how the path forwards traffic.
If port 8000 is already in use on your laptop or DGX device, choose a free port, such as 8001:
PORT=8000 to PORT=8001.http://localhost:8001 instead of http://localhost:8000.All supplied Intermediate scripts use Docker host networking. The PORT value sets vLLM's port on the DGX device, and Sync forwards the same port to your laptop. With PORT=8001, vLLM listens at 127.0.0.1:8001 on the DGX device.
For a recipe with --network host, change the server's --port 8000 to an unused port, such as --port 8001. Use that port for the DGX-local API checks and as the remote port in the SSH tunnel. If you use OpenShell, also update the port in its provider, network policy, and client settings.
If the selected recipe maps DGX host port 8000 and it is already in use, change the host-side port in the recipe's docker run -p mapping (the value to the left of the colon) to an unused port, such as 8001. Keep the container-side port and vllm serve --port unchanged. Use the new DGX host port for the device-local API checks and as <dgx-host-port> in the SSH tunnel command. Use the selected <laptop-port> in requests sent through the tunnel.
For tunnel-only access, bind the DGX host-side port to 127.0.0.1; publish it on other network interfaces only with deliberate access controls.
In the Advanced tab's Step 7 tunnel command, the leftmost port is the laptop port and the rightmost port is the DGX host port. If only the laptop port is busy, change the leftmost port and use it in the laptop client's API URL.
NOTE
Unified memory (UMA). On hardware platforms with unified memory, GPU and CPU share memory dynamically. Some applications have not yet been updated for UMA, so you may hit memory issues even within capacity. If that happens, manually flush the buffer cache:
sudo sh -c 'sync; echo 3 > /proc/sys/vm/drop_caches'
NOTE
Monitoring GPU memory with UMA. Because of unified memory, nvidia-smi --query-gpu memory fields report N/A. Use plain nvidia-smi instead.