Basic idea
Configure three DGX Spark systems in a ring topology for high-speed inter-node communication using 200GbE direct QSFP connections. This setup enables distributed workloads across three DGX Spark nodes by establishing network connectivity and configuring SSH authentication.
What you'll accomplish
You will physically connect three DGX Spark devices with QSFP cables, configure network interfaces for cluster communication, and establish passwordless SSH between nodes to create a functional distributed computing environment.
What to know before starting
- Basic understanding of distributed computing concepts
- Working with network interface configuration and netplan
- Experience with SSH key management
Prerequisites
- Three DGX Spark systems
- Three QSFP cables for direct 200GbE connection between the devices in a ring topology. Use recommended cable or similar.
- SSH access available to all systems
- Root or sudo access on all systems:
sudo whoami - The same username on all systems
- Update all systems to the latest OS and Firmware. Refer to the DGX Spark documentation https://docs.nvidia.com/dgx/dgx-spark/os-and-component-update.html
Ancillary files
This playbook's files can be found here on GitHub
- discover-sparks.sh script for automatic node discovery and SSH key distribution
- Cluster setup scripts for automatic network configuration, validation and running NCCL sanity test
Time & risk
-
Duration: 1 hour including validation
-
Risk level: Medium - involves network reconfiguration
-
Rollback: Network changes can be reversed by removing netplan configs or IP assignments
-
Last Updated: 3/19/2026
- First publication