---
title: "Install and Use NVIDIA PAIR with llama.cpp — Set Up the PAIR App"
canonical: "https://build.nvidia.com/rtx-spark/pair/app-setup.md"
---

# Step 1. Install and open PAIR

These instructions target **PAIR 1.0.0**, including llama.cpp support. Install
that version on each desktop system that will join the cluster.

Open the [NVIDIA PAIR download page](https://www.nvidia.com/en-us/ai-on-rtx/personal-ai-router/),
select **Download PAIR**, then choose the operating system and architecture.
Check that the downloaded filename contains `1.0.0`. If the download page
provides an older version, use the matching package from the
[PAIR releases page](https://github.com/NVIDIA/Personal-AI-Router/releases).

| System | Package filename |
| --- | --- |
| RTX Spark or another Windows ARM64 system | `NVPAIR-Setup-1.0.0-arm64.exe` |
| Windows on an Intel or AMD processor | `NVPAIR-Setup-1.0.0-x64.exe` |
| DGX Spark or another Debian/Ubuntu ARM64 system | `NVPAIR-Setup-1.0.0-arm64.deb` |
| Debian/Ubuntu on an Intel or AMD processor | `NVPAIR-Setup-1.0.0-amd64.deb` |
| Mac with Apple silicon | `NVPAIR-Setup-1.0.0-arm64.dmg` |
| Mac with an Intel processor | `NVPAIR-Setup-1.0.0-x64.dmg` |

## Windows

1. Download the Windows installer that matches the system's architecture.
2. Run the installer and approve the operating-system and firewall prompts.
3. Open **NVIDIA PAIR** from the Start menu.

RTX Spark uses the native Windows ARM64 installer and the PowerShell examples
below.

## Debian or Ubuntu

Open a terminal in the directory that contains the downloaded package, then
run:

```bash
sudo apt install "./NVPAIR-Setup-VERSION-ARCH.deb"
```

Replace `VERSION` and `ARCH` with the values in the downloaded filename. Then
open **NVIDIA PAIR** from the desktop application menu.

## macOS

Use macOS 13 or later. Choose ARM64 for Apple silicon and x64 for an Intel Mac.

1. Open the downloaded `.dmg`.
2. Drag **NVIDIA PAIR** to **Applications**.
3. Open **NVIDIA PAIR** from **Applications**.

Upgrading the macOS app preserves downloaded models.

PAIR opens to its setup window or **Overview** when started.

# Step 2. Complete first-run setup

When you first open PAIR, it shows the engines that it can install: Ollama,
LM Studio, and llama.cpp. Available engines can already be selected.

1. Review the available engines.
2. For the RTX Spark example, select **llama.cpp** and clear the other selections
unless you also want those engines. On other systems, choose any supported
engine, or skip engines that you manage separately.
3. Finish setup and wait for the selected engine to report that it is running.

The first start can take longer while PAIR starts its background services. If
**Overview** still shows **Loading...** after one or two minutes, open
**Settings → Service**, then use the **Troubleshooting** tab.

You can return to engine settings by selecting a node in **Overview**. Use
**Settings → Cluster** to pair systems at any time.

# Step 3. Pair systems

Start PAIR on each system. Confirm that the systems are on the same trusted
local network.

1. On the first system, select **Add node** in the top-right
toolbar. You can also open **Settings → Cluster** and use **Available nodes
to add**.
2. Select a discovered system. If PAIR does not find it, add the system by IP
address.
3. PAIR shows a six-digit PIN and sends an invitation.
4. On the invited system, accept the **Cluster invitation** and enter the PIN.
5. Open **Settings → Cluster** and check that the peer appears under
**Connected nodes**. You can also check the node list in **Overview**.
6. Repeat these steps from any cluster member to add more systems.

The peer appears as a connected node when pairing is complete. With one system,
continue to step 4 without pairing.

# Step 4. Start an engine and add a model

Repeat these steps on every system that should serve the model:

1. Select the system in **Overview** to open its engine settings.
2. Install **llama.cpp**, **Ollama**, or **LM Studio** if needed. Installation
starts the engine; if it is stopped, use its switch to start it.
3. Expand the engine and select **Add model**.
4. Select a model, select **Download**, and wait for the download to finish.
5. Load the model if the engine requires a separate load step.

A system can serve a request when it is online, the engine is running, and the
requested model is available there. To route requests across several systems,
add the same model to the same engine on each system that should serve it.
The downloaded model appears in that engine's model list.

## llama.cpp example

On RTX Spark, use PAIR's managed **llama.cpp** installation. PAIR downloads the
engine and CUDA runtime separately from the PAIR installer.

1. Under **llama.cpp**, select **Add model**.
2. Search for `ggml-org/gemma-3-1b-it-GGUF`. Typing filters the initial list;
press **Enter** or select the search button to search public Hugging Face
repositories if it is not listed.
3. Select the `Q4_K_M` entry and select **Download**.
4. Wait until `ggml-org/gemma-3-1b-it-GGUF:Q4_K_M` appears in the engine's model
list. Use **Load** to load it before testing, or allow the first request to
load it.

This small model is an example; you can choose another compatible GGUF model
that fits the system's memory. PAIR's llama.cpp catalog offers verified
`Q4_K_M` entries from public repositories. Model IDs include both the repository
and quantization: `owner/repository:quantization`. Use the exact ID when sending
a request; an Ollama tag such as `gemma3:1b` is a different identifier.

Use **Eject** to release a loaded model's memory while keeping its download.
Managed llama.cpp models also sleep after five minutes without inference work;
the next request wakes the model. Some GPU memory can remain allocated by the
engine.

# Step 5. Copy the local endpoint

Applications connect to PAIR on the system where they run. They do not connect
directly to an engine or to the system that will run the request.

1. Select **Endpoints** in PAIR's top toolbar.
2. In the **API endpoints** window, find the engine that you prepared.
3. Copy its `http://127.0.0.1:<port>` URL.

If the window says **No engines are running**, return to step 4 and start an
engine. The endpoint appears when that engine is running on any system in the
cluster, including a remote system.

> [!IMPORTANT]
> Copy the URL from **Endpoints**. PAIR puts a proxy on the engine's usual port
> and uses a separate port for the engine. llama.cpp defaults to proxy port
> `8080` and server port `8081`.

# Step 6. Send a test request

Use the endpoint from step 5 and the exact downloaded model ID from step 4.
The examples below use the OpenAI-compatible API to work with all three engines.

## Choose a request style

PAIR passes each request to the selected engine without rewriting it.

| Endpoint | OpenAI `/v1/chat/completions` | Ollama `/api/chat` |
| --- | --- | --- |
| Ollama | Works | Works |
| LM Studio | Works | Not available |
| llama.cpp | Works | Not available |

If you are unsure, use the OpenAI-style request. It works with every supported
engine.

## Windows PowerShell: llama.cpp example

Open PowerShell on the system running PAIR. Set `$pairBaseUrl` to the endpoint
you copied. With llama.cpp's default proxy port, list the available model IDs:

```powershell
$pairBaseUrl = "http://127.0.0.1:8080"
(Invoke-RestMethod -Uri "$pairBaseUrl/v1/models").data | Select-Object id
```

Confirm that the output includes the model you downloaded. Set `$pairModel` to
its exact `id`, then send the request:

```powershell
$pairModel = "ggml-org/gemma-3-1b-it-GGUF:Q4_K_M"
$pairRequest = @{
model = $pairModel
messages = @(@{
role = "user"
content = "Tell me a short story about a dog who learns to skateboard."
})
stream = $false
} | ConvertTo-Json -Depth 5
$pairResponse = Invoke-RestMethod -Uri "$pairBaseUrl/v1/chat/completions" -Method Post -ContentType "application/json" -Body $pairRequest
$pairResponse.choices[0].message.content
```

The response prints the model's answer. To use Ollama or LM Studio in
PowerShell, use that engine's endpoint and an exact model ID returned by its
`/v1/models` endpoint.

## Linux or macOS: OpenAI-style request

In a terminal, replace `<PAIR_BASE_URL>` with the endpoint from step 5 and list
its models:

```bash
curl <PAIR_BASE_URL>/v1/models
```

Copy the exact `id` from the response. Replace `<MODEL_NAME>` with that ID and
`<PAIR_BASE_URL>` with the same endpoint in the request below:

```bash
curl <PAIR_BASE_URL>/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "<MODEL_NAME>",
"messages": [
{
"role": "user",
"content": "Tell me a short story about a dog who learns to skateboard."
}
]
}'
```

## Linux or macOS: Ollama-style request

Use this request only with an Ollama endpoint. The `-N` option shows the
response as it streams.

```bash
curl -N <PAIR_BASE_URL>/api/chat \
-H "Content-Type: application/json" \
-d '{
"model": "<MODEL_NAME>",
"messages": [
{
"role": "user",
"content": "Tell me a short story about a dog who learns to skateboard."
}
]
}'
```

Open **Overview**, select the **Jobs** filter, and read **Ran on** or
**Running on** on the job card. A response and a job card show that PAIR routed
the request. Send several independent requests to observe routing across systems
that have the same engine and exact model ID. One request runs on one system.

# Step 7. Connect an application

Set the application's base URL to the endpoint from step 5. Select a model
that you added in step 4.

For an OpenAI-compatible SDK or application that appends the API path, include
`/v1` in the base URL. For example, llama.cpp uses
`http://127.0.0.1:8080/v1` with the default proxy port. A direct HTTP request uses
the complete path, such as `http://127.0.0.1:8080/v1/chat/completions`. An
Ollama-native client uses the host URL, such as `http://127.0.0.1:11434`.

PAIR accepts requests only from the system where it is running. The proxy uses
plaintext HTTP on loopback. A network request to an address such as
`http://some-node:11434` returns `403`.

Install PAIR on the system where you use the application. Join that system to
the cluster, then use its local endpoint. The system does not need a GPU or an
engine when another cluster system can serve the request.

PAIR does not provide a network-reachable inference endpoint. Configuring an
engine to listen on the network is outside PAIR and creates security exposure
that you must manage separately.

## Port reference

PAIR's proxy uses the port that an engine normally uses. Engines run behind the
proxy on separate server ports.

| Service | Default port |
| --- | --- |
| Ollama-compatible proxy | `11434` |
| Ollama engine behind PAIR | `11435` and upward |
| LM Studio / OpenAI-compatible proxy | `1234` |
| LM Studio engine behind PAIR | `1235` and upward |
| llama.cpp / OpenAI-compatible proxy | `8080` |
| Managed llama.cpp server behind PAIR | `8081` |

If `OLLAMA_HOST` names a different local loopback address, PAIR also serves
that address when the port is free. PAIR does not use a remote or HTTPS
`OLLAMA_HOST` value.

PAIR uses these ports to communicate between cluster systems:

| Port | Purpose |
| --- | --- |
| `5353/udp` | Local-network discovery through mDNS |
| `14318` | Node hardware and model inventory |
| `14319` | Service-error synchronization |
| `14320` | Workload propagation |
| `14321` | Pairing and cluster membership |
| `14322` | Model list served to cluster peers |
| `14323` | Cluster-scoped remote engine control |

The Windows installer adds the firewall rules. If a Linux firewall is
restrictive, allow these ports between trusted cluster systems.

# Next steps

## Change ports or engine launch options

1. Open **Overview** and expand **Engine settings** on the local system's card.
2. Expand the engine's **Settings**.
3. Edit **Proxy port**, **Server port**, or the engine arguments, then select
**Apply**. Confirm the restart if PAIR requests it.

PAIR applies the change as one operation and restores the new values when it
next starts. Supported paired systems can also be edited from another cluster
member. If you change a proxy port, update the application's base URL.
**Endpoints** always shows the current URL.

Keep managed llama.cpp in router mode. Do not add `-m` or `-hf` model-selection
arguments; download and load models with PAIR's model controls instead. See
[PAIR engine settings](https://github.com/NVIDIA/Personal-AI-Router/blob/feature/tui-llamacpp/docs/engine-settings.mdx)
for supported argument notation and settings.

For browser applications, configure only the required CORS origins on the system
running the engine. Managed llama.cpp starts without cross-origin browser
permission, and PAIR follows the engine's policy. Native clients such as
PowerShell do not need a CORS setting.

If another application uses a port that PAIR does not manage, choose a
different port in PAIR or stop the other application. Then restart the service
from **Settings → Service**.

## Use an engine's command line

PAIR-installed engines are normal installations, but their binaries are not in
`PATH` and their server ports differ from the usual defaults.

PAIR-managed Ollama locations:

| Platform | Path |
| --- | --- |
| Windows | `%LOCALAPPDATA%\Nvidia Corporation\Personal AI Router\engine-bin\ollama\ollama.exe` |
| Linux | `~/.config/Nvidia Corporation/Personal AI Router/engine-bin/ollama/bin/ollama` |
| macOS | `~/Library/Application Support/Nvidia Corporation/Personal AI Router/engine-bin/ollama/Ollama.app/Contents/Resources/ollama` |

On Linux, the table shows the default location. If `XDG_CONFIG_HOME` is set,
PAIR uses it instead of `~/.config`. The command below handles either location:

```bash
pair_ollama_dir="${XDG_CONFIG_HOME:-$HOME/.config}/Nvidia Corporation/Personal AI Router/engine-bin/ollama"
LD_LIBRARY_PATH="$pair_ollama_dir/lib/ollama" OLLAMA_HOST=127.0.0.1:11435 "$pair_ollama_dir/bin/ollama" list
```

Windows PowerShell example:

```powershell
$ollama = "$env:LOCALAPPDATA\Nvidia Corporation\Personal AI Router\engine-bin\ollama\ollama.exe"
$env:OLLAMA_HOST = "127.0.0.1:11435"
& $ollama list
```

LM Studio installs to its standard location:

```bash
~/.lmstudio/bin/lms status
```

On Windows, the executable is `%USERPROFILE%\.lmstudio\bin\lms.exe`.

Set `OLLAMA_HOST` to the engine's **Server** value when you want to inspect the
local engine. Otherwise, the CLI uses proxy port `11434` and returns the
cluster-wide view. The proxy shows what the cluster can serve. The server port
shows what is on that system.