Skip to main content
NVIDIA
Explore
Models
Skills
Blueprints
GPUs
Docs
Help Center
Getting Started
  1. Create and verify your account to unlock full access to NVIDIA NIM APIs.
ResourcesDeveloper ForumsContact Support
FAQs
    View All Playbooks
    View All Playbooks

    onboarding

    • Set Up Local Network Access
    • Open WebUI with Ollama

    data science

    • Single-cell RNA Sequencing
    • Portfolio Optimization
    • CUDA-X Data Science
    • Build Knowledge Graphs with txt2kg
    • Optimized JAX

    tools

    • DGX Dashboard
    • RAG Application in AI Workbench
    • Set up Tailscale on Your Spark
    • VS Code
    • Connect Three DGX Spark in a Ring Topology
    • Connect Multiple DGX Spark through a Switch

    fine tuning

    • FLUX.1 Dreambooth LoRA Fine-tuning
    • LLaMA Factory
    • Fine-tune with NeMo
    • Fine-tune with Pytorch
    • Unsloth on DGX Spark

    use case

    • Run Hermes Agent with a Local LLM
    • cuTile Kernels
    • CLI Coding Agent
    • Run NemoClaw with a Local LLM
    • 🦞 Set Up Example NemoClaw Agents 🦞
    • Live VLM WebUI
    • Install and Use Isaac Sim and Isaac Lab
    • Vibe Coding in VS Code
    • Build and Deploy a Multi-Agent Chatbot
    • Connect Two Sparks
    • NCCL for Multiple Sparks
    • Build a Video Search and Summarization (VSS) Agent
    • Spark & Reachy Photo Booth
    • Secure AI Agents with OpenShell
    • Run OpenClaw with a Local LLM

    inference

    • Generate Images and Videos with ComfyUI
    • Serve LLMs with vLLM
    • Speculative Decoding
    • Run models with llama.cpp on DGX Spark
    • Nemotron Model Family on DGX Spark
    • Serve LLMs with SGLang
    • TRT LLM for Inference
    • Quantize Models to NVFP4 with NVIDIA Model Optimizer
    • Multi-modal Inference
    • NIM on Spark
    • LM Studio on DGX Spark

    NCCL for Multiple Sparks

    30 MIN

    Install and test NCCL on two, three, or four Sparks

    • DGX
    • Spark
    OverviewOverviewRun on two SparksRun on two SparksRun on three SparksRun on three SparksRun on four SparksRun on four SparksTroubleshootingTroubleshooting

    Basic idea

    NCCL (NVIDIA Collective Communication Library) enables high-performance GPU-to-GPU communication across multiple nodes. This walkthrough sets up NCCL for multi-node distributed training on two, three, or four DGX Spark systems with Blackwell architecture. You'll configure networking, build NCCL from source with Blackwell support, and validate communication between nodes.

    What you'll accomplish

    You'll have a working multi-node NCCL environment that enables high-bandwidth GPU communication across DGX Spark systems for distributed training workloads, with validated network performance and proper GPU topology detection.

    What to know before starting

    • Working with Linux network configuration and netplan
    • Basic understanding of MPI (Message Passing Interface) concepts
    • SSH key management and passwordless authentication setup

    Prerequisites

    • Two, three, or four DGX Spark systems
    • Completed the matching connection playbook for your node count:
      • 2 Sparks — Connect Two Sparks
      • 3 Sparks — Connect Three Sparks
      • 4 Sparks — Connect Multiple Sparks through a Switch
    • NVIDIA driver installed: nvidia-smi
    • CUDA toolkit available: nvcc --version
    • Root/sudo privileges: sudo whoami

    Time & risk

    • Duration: 30 minutes for setup and validation
    • Risk level: Medium - involves network configuration changes
    • Rollback: The NCCL & NCCL Tests repositories can be deleted from DGX Spark
    • Last Updated: 12/15/2025
      • Use nccl latest version v2.30.7-1

    Resources

    • NCCL Documentation
    • DGX Spark Documentation
    • DGX Spark Forum
    • DGX Spark User Performance Guide
    Terms of Use
    Privacy Policy
    Your Privacy Choices
    Contact

    Copyright © 2026 NVIDIA Corporation