Serve LLMs with vLLM
30 MIN
High-throughput serving for 30+ models, with continuous batching and an OpenAI-compatible API
Agent-ready models
Agent-ready models are tuned for agentic workloads — tool calling, reasoning traces, and long multi-turn sessions. Use this tab to pick a recommended model for your hardware platform and follow the launch guidance below.
Follow the launch recipe below for your platform to set up the recommended model.
Recommendations by hardware platform
| Hardware platform | Recommended agent-ready model | MODEL_HANDLE | Recipe |
|---|---|---|---|
| DGX Spark | Agent-ready Qwen3.6-35B-A3B (NVFP4) | nvidia/Qwen3.6-35B-A3B-NVFP4 | Launch recipe |
| DGX Station | DeepSeek-V4-Flash | deepseek-ai/DeepSeek-V4-Flash | Launch recipe |
| RTX PRO | Qwen3.6 27B | nvidia/Qwen3.6-27B-NVFP4 | Launch recipe |
Verify your server
After you launch a recipe above, confirm startup using Watch startup in the Instructions tab, then test with Step 5. Test the API.
Next steps
- General serving workflow: Docker setup, health checks, and API testing — see the Instructions tab
- Scale out: multi-node serving on multi-node capable hardware — see the Multi-node serving tab