Connect Multiple DGX Sparks for Distributed Workloads

This is the recommended path for most users

If you are working with up to four devices, this is the recommended path.

You can see how it works in this demo video.

If you want to cluster more than four devices, then follow and adapt the Configure Manually tab.

Make sure the devices are properly connected

Follow the instructions in the Cable the Devices tab.

Install NVIDIA Sync on your laptop

Install NVIDIA Sync on your Windows, macOS, or Ubuntu laptop.

• Windows: Open the .exe file and follow the setup steps.
• macOS: Open nvidia-sync.dmg, move NVIDIA Sync to the Applications folder, and open it.
• Ubuntu: Follow the NVIDIA Sync install guide.

Add each DGX Spark to NVIDIA Sync

Make sure your laptop can reach each DGX Spark on the LAN.

For each device:

1. Open NVIDIA Sync and select Add New.
2. Pick the device if its mDNS name appears. If it does not, select Add device manually.
3. Enter the device name or IP address, user name, and password.
4. Select Add.

Success: Each DGX Spark should now appear in NVIDIA Sync.

Start the NVIDIA Sync Cluster Assistant

1. Open the NVIDIA Sync Settings and select Cluster Assistant.
2. Then select Add New Cluster and give it a name.
3. Next, select the devices that you physically connected.

Resolve any issues NVIDIA Sync finds in the device and link checks

After you select the devices, NVIDIA Sync will check the devices and physical connections are appropriately set up.

These checks are divided into stages.

• SSH access to each device
• The hardware and system software
• sudo privileges and password requirement
• User names and IDs across devices
• Link configuration across devices
• Existing compliant network plan on each device

If a check fails, fix the issue and retry.

1. NVIDIA Sync should already have SSH access because you added the devices
2. If one of the selected devices isn't a Spark, you must remove it
3. If any device needs a DGX OS update, you must update it and retry
4. If sudo requires a password on a device, you must enter it for temporary privileges
5. (optional) If the user name, ID and group ID is different across the devices, you may want to make them the same - not necessary but can simplify downstream workloads
6. If a cable is not properly connected, fix and retry
7. (optional) If the negotiated speed of each physical link is too slow, fix it and retry

Success: All checks pass and you are asked to confirm the network plan

NVIDIA Sync sets up inter-device SSH

Once the network plan is confirmed, NVIDIA Sync will set up key-based SSH with an alias between the devices on the ConnectX-7 network for process management across devices.

This SSH does not use the LAN that you use for SSH connections to the devices. It goes across the physical connections.

This step can take a few minutes. If one device times out after five minutes, try the step again.

Success: You see the success screen with steps to move on to configuring an actual workload on the cluster. The ConnectX-7 network and inter-device SSH are now ready.

Save the cluster details to a text file

Save the cluster details to a text file when NVIDIA Sync shows the success page.

1. Copy the network details shown on the success page. If your version of NVIDIA Sync shows a Copy button, select it.
2. Save the details in a file for later use.
3. To continue, use the example workloads in Next steps below. If your version of NVIDIA Sync shows See Example Workloads, you can select it instead.

Next steps

• Learn how to use the NVIDIA Sync cluster assistant
• Set up NCCL on the cluster
• Run a fine-tuning workload with PyTorch
• Set up vLLM on the cluster