---
title: "Nemotron Model Family on DGX Spark — Run Nemotron Nano"
canonical: "https://build.nvidia.com/spark/nemotron/instructions-nano.md"
---

# Step 1. Overview and prerequisites

This tab serves **NVIDIA Nemotron-3-Nano** on a single DGX Spark with **vLLM**, using the upstream multi-arch image **`vllm/vllm-openai:v0.20.0`**. The server exposes an OpenAI-compatible HTTP API on port **8000**. The example uses the multimodal **Nemotron-3-Nano Omni** weights (text, image, audio, and video), so the launch command installs the optional audio packages and enables the multimodal limits.

DGX Spark ships a single Grace–Blackwell GB10 GPU with **128 GB of unified memory**—the batch, context, and cache choices below assume that footprint.

This tab covers the **Spark-specific setup**. For everything not covered here—**API examples, reasoning mode, and video tuning**—follow the general Nemotron instructions and the model card.

**Requirements**

- DGX Spark with GB10 and sufficient disk for weights, caches, and the container image
- NVIDIA Container Toolkit and Docker with GPU access
- Local **Nemotron-3-Nano Omni** weights on disk (point `WEIGHTS` at that directory in Step 3)

---

# Step 2. Pull the vLLM container image

Pull the upstream multi-arch vLLM **v0.20.0** image. Docker automatically pulls the **arm64** variant on Spark.

```bash
docker pull vllm/vllm-openai:v0.20.0
```

---

# Step 3. Launch the vLLM server on Spark

Point `WEIGHTS` at your local Nemotron-3-Nano Omni weights directory, then start the server. The base image does **not** include audio packages, so the command installs them with `pip install vllm[audio]` before running `vllm serve`.

```bash
WEIGHTS=/path/to/nemotron-3-nano-omni-weights

docker run --rm -it \
--gpus all \
--ipc=host -p 8000:8000 \
--shm-size=16g \
--name vllm-nemotron-omni \
-v "${WEIGHTS}:/model:ro" \
--entrypoint /bin/bash \
vllm/vllm-openai:v0.20.0 -c  \
"pip install vllm[audio] && vllm serve /model \
--served-model-name=nemotron_3_nano_omni \
--max-num-seqs 8 \
--max-model-len 131072 \
--port 8000 \
--trust-remote-code \
--gpu-memory-utilization 0.8 \
--limit-mm-per-prompt '{\"video\": 1, \"image\": 1, \"audio\": 1}' \
--media-io-kwargs '{\"video\": {\"fps\": 2,  \"num_frames\": 256}}' \
--allowed-local-media-path=/ \
--enable-prefix-caching \
--max-num-batched-tokens 32768 \
--reasoning-parser nemotron_v3 \
--enable-auto-tool-choice \
--tool-call-parser qwen3_coder"
```

**Flag rationale (summary)**

| Flag | Role |
| ---- | ---- |
| `--served-model-name=nemotron_3_nano_omni` | The `model` id clients pass to the API |
| `--max-num-seqs 8` | Conservative concurrency for memory headroom |
| `--max-model-len 131072` | Context window; reduce first if you hit OOM (see Step 5) |
| `--gpu-memory-utilization 0.8` | Fraction of unified memory vLLM may use |
| `--limit-mm-per-prompt` | Caps multimodal inputs per request (1 each of video, image, audio) |
| `--media-io-kwargs` | Video decode settings (2 fps, up to 256 frames) |
| `--allowed-local-media-path=/` | Lets the server read local media files by path |
| `--enable-prefix-caching` | Reuses shared prompt prefixes across requests |
| `--reasoning-parser nemotron_v3` | Parses Nemotron reasoning output |
| `--enable-auto-tool-choice` / `--tool-call-parser qwen3_coder` | Tool calling support |

**Key Spark-specific flags**

| Flag | Purpose | Spark guidance |
| ---- | ------- | -------------- |
| `--gpus all` | Select GPU | Spark has one GB10 GPU; `all` is equivalent to `device=0` |
| `--max-model-len` | Max context window | Start at `131072`; reduce if you hit OOM (see Step 5) |

---

# Step 4. Verify the server and test the API

In another terminal, confirm the server is ready and reports the served model:

```bash
curl -sS http://localhost:8000/v1/models | python3 -m json.tool
```

Once the model is listed, send a chat completion. Use the served model name from Step 3:

```bash
curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "nemotron_3_nano_omni",
"messages": [{"role": "user", "content": "New York is a great city because..."}],
"max_tokens": 100
}'
```

For multimodal requests, reasoning-mode prompting, and video tuning, follow the general Nemotron instructions and the model card.

---

# Step 5. Memory tuning on Spark

Spark uses unified LPDDR5X memory (~128 GB shared between CPU and GPU), not separate system + VRAM pools. If you hit OOM, use these two levers, in order of impact:

1. **Lower `--gpu-memory-utilization`** from `0.8` toward `0.70` to free memory back to the OS and re-enable weight prefetch. Cost: a smaller KV cache budget.
2. **Lower `--max-model-len`** to reduce KV cache allocation (for example, halving the context window halves the KV cache at `--max-num-seqs=1`).

Combined override for a tight-memory run:

```bash
--gpu-memory-utilization=0.70 \
--max-model-len=32768 \
```

---

# Step 6. Cleanup

Stop the running container with `Ctrl+C`, or from another terminal:

```bash
docker stop vllm-nemotron-omni
```

Remove the image only if you want to reclaim disk:

```bash
docker rmi vllm/vllm-openai:v0.20.0
```

Delete the local weights directory only if you no longer need it.

For the larger **Nemotron Super** deployment on the same hardware, use the **Run Nemotron Super** tab in this playbook.