--- title: "Install and Use NVIDIA PAIR with llama.cpp — Overview" canonical: "https://build.nvidia.com/rtx-spark/pair/overview.md" --- # Basic idea NVIDIA PAIR connects computers on your local network and presents Ollama-compatible and OpenAI-compatible proxy endpoints to applications and agents. It automatically routes each independent AI inference request to an eligible computer according to engine availability, model availability, and current workload. Run PAIR on one computer for local inference, or run it on several computers to handle more requests. This playbook covers PAIR **1.0.0**, including managed **llama.cpp** support and the redesigned terminal interface. Ollama and LM Studio remain available. PAIR is useful for workloads such as multi-agent applications that make several requests at once. PAIR is designed to keep prompts and responses on your local network when the application, model source, engine, and paired computers are all local. > [!IMPORTANT] > PAIR sends each request to one system. It does not combine GPU memory, join > GPUs into one larger GPU, or split a model or request across systems. # What you'll accomplish You'll install PAIR, prepare an engine and model, and send a test request through PAIR's local endpoint. You can also pair trusted systems and route independent requests to systems that have the requested model. # What to know before starting **Required:** - One compatible system on which to install PAIR. - llama.cpp, Ollama, or LM Studio, plus a model, on at least one system that will serve requests. - A trusted local network when you pair systems. The six-digit PIN is a short-lived setup code, not a long-term credential. **Optional:** - Two or more systems to route requests across a local cluster. - A graphical desktop. Use the terminal interface on a headless system or over SSH. Use one PAIR interface at a time. Do not run the desktop application and the terminal interface on the same system. They start competing services and can conflict over ports, engines, and settings. PAIR accepts requests only from the local system. Install PAIR on the system where your application runs. That system can use an engine on another cluster node, so it does not need its own GPU or inference engine. Read the [PAIR security policy](https://github.com/NVIDIA/Personal-AI-Router/blob/main/SECURITY.md) before you use PAIR on a shared or untrusted network. # Supported hardware platforms Use the matrix below to confirm your hardware platform, operating system, memory, and whether you can use it in a PAIR cluster. | Hardware platform | OS | Memory | Multi-node capable hardware | | :---- | :---- | :---- | :---- | | **RTX Spark** | Windows 11 on ARM | Unified memory; capacity depends on the system | ✅ (PAIR cluster) | | **DGX Spark** | DGX OS (Linux) | 128 GB Unified Memory | ✅ (PAIR cluster) | | **GeForce RTX** | Windows 11 or Linux | Depends on the GPU and model | ✅ (PAIR cluster) | | **RTX PRO** | Windows 11 or Linux | Depends on the GPU and model | ✅ (PAIR cluster) | | **Mac (Apple silicon or Intel)** | macOS 13 or later | Depends on the Mac and model | ✅ (PAIR cluster) | NVIDIA RTX Spark, DGX Spark, GeForce RTX, RTX PRO, and compatible Macs are the hardware platforms covered by this playbook. GeForce RTX requires a 20 Series or newer GPU, and RTX PRO requires a Turing or newer GPU. PAIR can also pair compatible systems that run the supported operating systems below. | Support | Details | | :---- | :---- | | Operating systems | Windows 11, Linux, and macOS 13 or later | | Architectures | x64 and ARM64 packages for Windows, Linux, and macOS. RTX Spark uses Windows ARM64; DGX Spark uses Linux ARM64. | | Installers | Windows `.exe`, Linux `.deb`, and macOS `.dmg`. Build from source for other Linux distributions. | | Mixing systems | Windows, Linux, and macOS systems can pair with each other. | | Inference engines | llama.cpp, Ollama, and LM Studio | Open the [NVIDIA PAIR download page](https://www.nvidia.com/en-us/ai-on-rtx/personal-ai-router/) and select your operating system and architecture. Check that the downloaded package is for **1.0.0**; use the [PAIR releases page](https://github.com/NVIDIA/Personal-AI-Router/releases) to check the version and release notes. The app setup tab lists the installer choices. These instructions use native Windows PowerShell on Windows, with no WSL requirement. ## Why memory matters Each system serving a request needs enough memory for the model weights, the model's context, and the engine's runtime overhead. A model's download size is not its total memory requirement. On RTX Spark, DGX Spark, and Apple silicon Macs, the CPU and GPU share unified memory; on systems with a discrete GPU, GPU VRAM and system RAM are separate resources. Leave room for the operating system and other applications. Use your system's memory information and the model's requirements when choosing a model. PAIR's GPU-memory display can understate available memory on Windows systems with unified memory. Pairing systems does not add their memory together to fit one larger model. # Prerequisites **Hardware requirements** - One compatible system for local inference. - Two or more compatible systems on the same trusted local network for cluster routing. **Software requirements** - The PAIR 1.0.0 package for each participating system's OS and architecture. - llama.cpp, Ollama, or LM Studio running on each system that will serve requests. - A model downloaded on at least one system that will serve requests. - Internet access and free disk space for installer, engine, and model downloads. PAIR can run on a supported system even when an engine cannot. Each engine has its own requirements for the operating system, GPU, and drivers. Each model also needs enough memory to load. Check the engine documentation before you expect a system to serve a model. ## Choose an engine and model For the RTX Spark example in this playbook, select **llama.cpp** in PAIR and let PAIR install it. The 1.0.0 release adds llama.cpp with CUDA support for RTX Spark's Windows ARM64 platform. Choose Ollama or LM Studio when you need their model format or client workflow and they support your system. | Engine | Model selection | Request style | | --- | --- | --- | | llama.cpp | Compatible GGUF models from Hugging Face, such as `ggml-org/gemma-3-1b-it-GGUF:Q4_K_M` | OpenAI-compatible | | Ollama | Exact Ollama model names, such as `llama3.2` | OpenAI-compatible or Ollama-native | | LM Studio | A model available through LM Studio; copy its advertised ID | OpenAI-compatible | For llama.cpp, the example ID consists of a Hugging Face owner (`ggml-org`), repository (`gemma-3-1b-it-GGUF`), and quantization (`Q4_K_M`). Use the exact model ID shown by PAIR or returned by `/v1/models` in requests; do not substitute a local GGUF filename or a display name. # Ancillary files No extra files are required. # Time & risk - **Estimated time:** About 10 minutes, plus engine and model download time. - **Risk level:** Low to medium. PAIR installs software, downloads models, and enables communication between trusted local systems. - **Rollback:** Remove paired systems, uninstall PAIR, and remove engines or models you no longer need. - **Last updated:** 10/06/2026. For a graphical setup, open **Set Up the PAIR App**. For a headless or SSH setup, open **Set up PAIR with Terminal**.