---
title: "Serve LLMs with vLLM on One Device"
publisher: "nvidia"
type: "playbook"
updated: "2026-08-06T18:38:21.718Z"
description: "A locally accessible model API from one DGX Spark or Station"
canonical: "https://build.nvidia.com/spark/vllm.md"
---

# Basic idea

[vLLM](https://vllm.ai/) is a high-performance inference engine for serving LLMs at datacenter scale.
However, getting the desired performance requires significant background knowledge and configuration experience, especially when setting up on a local system.

This playbook shows you two paths to set up vLLM and configure the endpoint on a remote DGX Spark or DGX Station ([for a Spark cluster, go here](https://build.nvidia.com/playbooks/vllm-cluster)). 

# What you'll accomplish

- **Intermediate:** Manually prepare a recommended model on the remote device and configure the vLLM container as an NVIDIA Sync Custom App.
- **Advanced:** Get under the hood and do everything manually to learn about vLLM and optimize model performance.

# What to know before starting

- How to use [NVIDIA Sync](https://docs.nvidia.com/sync/latest/index.html). **Intermediate**
- How to read Docker and Hugging Face CLI commands. **Intermediate · Advanced**
- How to read and edit a Bash launch script. **Intermediate · Advanced**
- How to choose a model recipe and understanding model size and precision. **Advanced**
- How to configure `docker run` and `vllm serve` arguments. **Advanced**

# Supported hardware platforms

Check the table below to confirm which path this playbook supports for your hardware.

| Hardware platform | OS | Memory | One device |
| :---- | :---- | :---- | :----: |
| **DGX Spark** | DGX OS (Linux) | 64 GB or 128 GB Unified Memory | ✅ |
| **DGX Station** | Ubuntu (Linux) | Large HBM + Grace DRAM | ✅ |

# Prerequisites

**Hardware requirements**

- One DGX Spark or DGX Station. 

**Software and access requirements**

- [NVIDIA Sync installed](https://docs.nvidia.com/sync/latest/getting-started.html#installation-and-onboarding) on your laptop, with the DGX device [connected](https://docs.nvidia.com/sync/latest/direct-connections.html#adding-a-device-for-a-direct-connection). **Intermediate** 
- Docker installed on the DGX device and [available to your user](https://docs.docker.com/engine/install/linux-postinstall/#add-your-user-to-the-docker-group). **Intermediate · Advanced**
- An [installed](https://huggingface.co/docs/huggingface_hub/main/en/guides/cli#standalone-installer-recommended) Hugging Face CLI on the remote device. **Intermediate · Advanced**
- SSH access to the remote device. **Advanced**

# Time & risk

- **Estimated time:** Intermediate varies between 15 and 30 minutes depending on network speed. Advanced takes about 30 minutes more to account for manual steps.
- **Risk level:** Low
- **Stop or rollback:** You can stop configuration flows at any time, and you can remove the container or model from the device manually.
- **Last Updated:** 10/19/2026
- Intermediate offers two models per device and memory size, with matching NVIDIA Sync launch scripts
- Advanced combines Spark and Station in one recipe workflow, with explanations of memory, scheduling, and model settings
- Advanced includes API and tool-call checks, manual launch scripts, and client setup for Hermes, OpenShell, and laptop access

## More

- [Intermediate](/spark/vllm/intermediate.md)
- [Advanced](/spark/vllm/advanced.md)
- [Troubleshooting](/spark/vllm/troubleshooting.md)