Basic idea
vLLM is a high-performance inference engine for serving LLMs at datacenter scale.
However, getting the desired performance requires significant background knowledge and configuration experience, especially when setting up on a local system.
This playbook shows you two paths to set up vLLM and configure the endpoint on a remote DGX Spark or DGX Station (for a Spark cluster, go here).
What you'll accomplish
- Intermediate: Manually prepare a recommended model on the remote device and configure the vLLM container as an NVIDIA Sync Custom App.
- Advanced: Get under the hood and do everything manually to learn about vLLM and optimize model performance.
What to know before starting
- How to use NVIDIA Sync. Intermediate
- How to read Docker and Hugging Face CLI commands. Intermediate · Advanced
- How to read and edit a Bash launch script. Intermediate · Advanced
- How to choose a model recipe and understanding model size and precision. Advanced
- How to configure
docker run and vllm serve arguments. Advanced
Supported hardware platforms
Check the table below to confirm which path this playbook supports for your hardware.
| Hardware platform | OS | Memory | One device |
|---|
| DGX Spark | DGX OS (Linux) | 64 GB or 128 GB Unified Memory | ✅ |
| DGX Station | Ubuntu (Linux) | Large HBM + Grace DRAM | ✅ |
Prerequisites
Hardware requirements
- One DGX Spark or DGX Station.
Software and access requirements
Time & risk
- Estimated time: Intermediate varies between 15 and 30 minutes depending on network speed. Advanced takes about 30 minutes more to account for manual steps.
- Risk level: Low
- Stop or rollback: You can stop configuration flows at any time, and you can remove the container or model from the device manually.
- Last Updated: 10/19/2026
- Intermediate offers two models per device and memory size, with matching NVIDIA Sync launch scripts
- Advanced combines Spark and Station in one recipe workflow, with explanations of memory, scheduling, and model settings
- Advanced includes API and tool-call checks, manual launch scripts, and client setup for Hermes, OpenShell, and laptop access
