Serve LLMs with vLLM Across DGX Sparks

Basic idea

If you want to run a workload or a model that won't fit on a single DGX Spark, you can connect two or more Sparks into a cluster and distribute the workload or model across the devices.

However, this takes a bit of set up and experience. Even if you have done it before, it can still involve a lot of trial and error.

This playbook shows you two paths to set up containerized vLLM to run a model distributed across the nodes (see here for a single node).

What you'll accomplish

Pick a path below to set up the cluster, distribute the model and make it available through an endpoint at localhost on your laptop.

• Intermediate: Use NVIDIA Sync to set up the cluster, then choose a model recipe and use NVIDIA provided scripts to set spark-vllm-docker to serve it (two Sparks).
• Advanced: Follow the manual instructions and use NVIDIA provided scripts (two Sparks).

What to know before starting

• Using NVIDIA Sync to connect to devices and launch an application. Intermediate
• Choosing a model recipe and understanding model size, precision, and hardware fit. Intermediate · Advanced
• Reading Docker and Hugging Face CLI commands. Intermediate · Advanced
• Reading shell scripts and vLLM launch options, including tensor parallelism. Advanced

Supported hardware platforms

The guided path supports two to four DGX Sparks. The command-line examples currently cover two.

The dashes mean this draft has no worked command-line example for those cluster sizes.

Prerequisites

Hardware requirements

• Two to four DGX Sparks with the necessary QSFP cables (switch required for four Sparks). All paths
• Two DGX Sparks for the current command-line examples. Intermediate · Advanced

Software and access requirements

• NVIDIA Sync installed on your laptop and each Spark added before using Cluster Assistant. Intermediate
• Docker installed and available to your user on each Spark. All paths
• An installed, authenticated Hugging Face CLI on each Spark. Intermediate · Advanced
• The uv package manager installed on the Spark you use for the head node. Intermediate · Advanced

Find model recipes

The two-Spark worked example uses qwen3.8-27b-nvfp4-dflash2.yaml from spark-vllm-docker to serve nvidia/Qwen3.8-27B-NVFP4.

Time & risk

• Estimated time: Plan for 30 minutes or longer when downloading a model for the first time. Cluster setup and download time vary with the number of devices and chosen model.
• Risk level: Medium; the procedure changes the cluster network configuration and launches processes on multiple Sparks.
• Stop or rollback: For Beginner, stop the model in NVIDIA Sync. For Intermediate and Advanced, stop the deployment on both Sparks from the head node before changing or deleting the cluster. Remove the cluster in NVIDIA Sync if you want to undo its network setup.
• Last Updated: 09/28/2026
  • Split the cluster workflow from the single-device vLLM playbook.