NCCL for Multiple Sparks

Basic idea

NCCL (NVIDIA Collective Communication Library) enables high-performance GPU-to-GPU communication across multiple nodes. This walkthrough sets up NCCL for multi-node distributed training on two, three, or four DGX Spark systems with Blackwell architecture. You'll configure networking, build NCCL from source with Blackwell support, and validate communication between nodes.

What you'll accomplish

You'll have a working multi-node NCCL environment that enables high-bandwidth GPU communication across DGX Spark systems for distributed training workloads, with validated network performance and proper GPU topology detection.

What to know before starting

• Working with Linux network configuration and netplan
• Basic understanding of MPI (Message Passing Interface) concepts
• SSH key management and passwordless authentication setup

Prerequisites

• Two, three, or four DGX Spark systems
• Completed the matching connection playbook for your node count:
  • 2 Sparks — Connect Two Sparks
  • 3 Sparks — Connect Three Sparks
  • 4 Sparks — Connect Multiple Sparks through a Switch
• NVIDIA driver installed: nvidia-smi
• CUDA toolkit available: nvcc --version
• Root/sudo privileges: sudo whoami

Time & risk

• Duration: 30 minutes for setup and validation
• Risk level: Medium - involves network configuration changes
• Rollback: The NCCL & NCCL Tests repositories can be deleted from DGX Spark
• Last Updated: 12/15/2025
  • Use nccl latest version v2.30.7-1