Nemotron Model Family on DGX Spark

Basic idea

Nemotron is NVIDIA’s open language model family built for real workloads: strong reasoning, tool use, and long-context chat in a stack you can run on your own hardware. Models in the line share a similar product shape—capable assistants you can serve behind an OpenAI-style API and integrate with agents, IDEs, or custom apps—while checkpoints differ in size, architecture (including efficient MoE designs), and the serving stack that fits them best.

The family lives in the open Nemotron project and on Hugging Face alongside documentation and cookbooks from NVIDIA and the community. This playbook is your on-ramp for running Nemotron on a single DGX Spark: you get a working inference endpoint, not a tour of every training or export option.

What you'll accomplish

You will deploy a production-style Nemotron inference server on DGX Spark with an OpenAI-compatible HTTP API, so you can send chat completions from curl, scripts, or client libraries. You will use checkpoints and runtimes that are known to work on Spark’s GB10 GPU and unified memory, with steps for verifying the server and (where the model supports it) reasoning-style behavior.

The instruction tabs are two supported ways to get there—different models and engines—picked to match what actually fits on one machine. You do not need both; choose the tab that matches the model you want.

What to know before starting

• Comfortable using a Linux terminal, environment variables, and running Docker with GPU access.
• Able to use Hugging Face Hub for downloads and, for gated assets, an account and token.
• Basic familiarity with REST APIs or curl for a quick smoke test.

Prerequisites

Hardware

• NVIDIA DGX Spark with GB10
• Enough disk for model weights, build artifacts or container layers, and caches (plan for tens of gigabytes minimum; large checkpoints need more)

Software (baseline)

• Network access to GitHub, Hugging Face, and (for containerized serving) container registries
• Current NVIDIA GPU driver stack appropriate for your Spark image

Choosing a tab (only split here)

• Run Nemotron Nano — You serve Nemotron-3-Nano on Docker with vLLM (vllm/vllm-openai:v0.20.0), pointing the container at local Nemotron-3-Nano Omni weights and running vllm serve on port 8000. You will want the NVIDIA Container Toolkit and Docker with GPU access.
• Run Nemotron Super — You serve the NVFP4 Nemotron-3-Super checkpoint on Docker with either vLLM (vllm/vllm-openai:cu130-nightly, reasoning-parser plugin, tuned env vars and flags) or TensorRT-LLM (nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc9, hf download into a local folder, extra-llm-api-config.yml, and trtllm-serve). You will want the NVIDIA Container Toolkit, Docker with GPU access, and a Hugging Face token where the model card requires it.

Time & risk

• Time: First-time setup usually on the order of tens of minutes to longer, dominated by downloads (weights, and for Super, the inference image). Exact times depend on bandwidth and cache state.
• Risk level: Low — work is in user space (Docker, weight directories, cache dirs); you are not asked to reimage the system.
• Rollback: Remove the model/weight directories you created, prune Docker images/containers, and clear Hugging Face cache only if you intend to reclaim disk.
• Last updated: 07/01/2026 — Switch Nemotron Nano tab to vLLM