---
title: "Connect Multiple DGX Sparks for Distributed Workloads — Configure with NVIDIA Sync"
canonical: "https://build.nvidia.com/playbooks/connect-multiple-sparks/beginner.md"
---

# This is the recommended path for most users

If you are working with up to four devices, this is the recommended path.

You can see how it works in [this demo video](https://www.youtube.com/watch?v=MehBUQtb9qM).

If you want to cluster more than four devices, then follow and adapt the Configure Manually tab.

# Step 1. Make sure the devices are properly connected

**Follow the instructions in the Cable the Devices tab.**

# Step 2. Install NVIDIA Sync on your laptop

**Install NVIDIA Sync on your Windows, macOS, or Ubuntu laptop.**

::spark-download

- **Windows:** Open the `.exe` file and follow the setup steps.
- **macOS:** Open `nvidia-sync.dmg`, move NVIDIA Sync to the Applications folder, and open it.
- **Ubuntu:** Follow the [NVIDIA Sync install guide](https://docs.nvidia.com/sync/latest/getting-started.html#installation-and-onboarding).

# Step 3. Add each DGX Spark to NVIDIA Sync

**Make sure your laptop can reach each DGX Spark on the LAN.**

For each device:

1. Open NVIDIA Sync and select **Add New**.
2. Pick the device if its mDNS name appears. If it does not, select **Add device manually**.
3. Enter the device name or IP address, user name, and password.
4. Select **Add**.

**Success**: Each DGX Spark should now appear in NVIDIA Sync.

# Step 4. Start the NVIDIA Sync Cluster Assistant

1. Open the NVIDIA Sync **Settings** and select **Cluster Assistant**.
2. Then select **Add New Cluster** and give it a name.
3. Next, select the devices that you physically connected.

# Step 5. Resolve any issues NVIDIA Sync finds in the device and link checks

After you select the devices, NVIDIA Sync will check the devices and physical connections are appropriately set up. 

These checks are divided into stages.

- SSH access to each device
- The hardware and system software
- `sudo` privileges and password requirement
- User names and IDs across devices
- Link configuration across devices
- Existing compliant network plan on each device

**If a check fails, fix the issue and retry.**

1. NVIDIA Sync should already have SSH access because you added the devices
2. If one of the selected devices isn't a Spark, you must remove it
3. If any device needs a DGX OS update, you must update it and retry
4. If `sudo` requires a password on a device, you must enter it for temporary privileges
5. (optional) If the user name, ID and group ID is different across the devices, you may want to make them the same - not necessary but can simplify downstream workloads
6. If a cable is not properly connected, fix and retry
7. (optional) If the negotiated speed of each physical link is too slow, fix it and retry

**Success**: All checks pass and you are asked to confirm the network plan

# Step 6. NVIDIA Sync sets up inter-device SSH

Once the network plan is confirmed, NVIDIA Sync will set up key-based SSH with an alias between the devices on the ConnectX-7 network for process management across devices. 

This SSH **does not** use the LAN that you use for SSH connections to the devices. 
It goes across the physical connections.

This step can take a few minutes. 
If one device times out after five minutes, try the step again.

**Success**: You see the success screen with steps to move on to configuring an actual workload on the cluster.
The ConnectX-7 network and inter-device SSH are now ready.

# Step 7. Save the cluster details to a text file

**Save the cluster details to a text file when NVIDIA Sync shows the success page.**

1. Copy the network details shown on the success page. If your version of NVIDIA Sync shows a **Copy** button, select it.
2. Save the details in a file for later use.
3. To continue, use the example workloads in **Next steps** below. If your version of NVIDIA Sync shows **See Example Workloads**, you can select it instead.

# Next steps

- [Learn how to use the NVIDIA Sync cluster assistant](https://docs.nvidia.com/sync/latest/cluster-assistant.html#nvidia-sync-cluster-assistant)
- [Set up NCCL on the cluster](https://build.nvidia.com/playbooks/nccl)
- [Run a fine-tuning workload with PyTorch](https://build.nvidia.com/spark/multi-sparks-distributed-finetuning)
- [Set up vLLM on the cluster](https://build.nvidia.com/spark/vllm)