Install and Use NVIDIA PAIR with llama.cpp

Install and open PAIR

These instructions target PAIR 1.0.0, including llama.cpp support. Install that version on each desktop system that will join the cluster.

Open the NVIDIA PAIR download page, select Download PAIR, then choose the operating system and architecture. Check that the downloaded filename contains 1.0.0. If the download page provides an older version, use the matching package from the PAIR releases page.

Windows

1. Download the Windows installer that matches the system's architecture.
2. Run the installer and approve the operating-system and firewall prompts.
3. Open NVIDIA PAIR from the Start menu.

RTX Spark uses the native Windows ARM64 installer and the PowerShell examples below.

Debian or Ubuntu

Open a terminal in the directory that contains the downloaded package, then run:

sudo apt install "./NVPAIR-Setup-VERSION-ARCH.deb"

Replace VERSION and ARCH with the values in the downloaded filename. Then open NVIDIA PAIR from the desktop application menu.

macOS

Use macOS 13 or later. Choose ARM64 for Apple silicon and x64 for an Intel Mac.

1. Open the downloaded .dmg.
2. Drag NVIDIA PAIR to Applications.
3. Open NVIDIA PAIR from Applications.

Upgrading the macOS app preserves downloaded models.

PAIR opens to its setup window or Overview when started.

Complete first-run setup

When you first open PAIR, it shows the engines that it can install: Ollama, LM Studio, and llama.cpp. Available engines can already be selected.

1. Review the available engines.
2. For the RTX Spark example, select llama.cpp and clear the other selections unless you also want those engines. On other systems, choose any supported engine, or skip engines that you manage separately.
3. Finish setup and wait for the selected engine to report that it is running.

The first start can take longer while PAIR starts its background services. If Overview still shows Loading... after one or two minutes, open Settings → Service, then use the Troubleshooting tab.

You can return to engine settings by selecting a node in Overview. Use Settings → Cluster to pair systems at any time.

Pair systems

Start PAIR on each system. Confirm that the systems are on the same trusted local network.

1. On the first system, select Add node in the top-right toolbar. You can also open Settings → Cluster and use Available nodes to add.
2. Select a discovered system. If PAIR does not find it, add the system by IP address.
3. PAIR shows a six-digit PIN and sends an invitation.
4. On the invited system, accept the Cluster invitation and enter the PIN.
5. Open Settings → Cluster and check that the peer appears under Connected nodes. You can also check the node list in Overview.
6. Repeat these steps from any cluster member to add more systems.

The peer appears as a connected node when pairing is complete. With one system, continue to step 4 without pairing.

Start an engine and add a model

Repeat these steps on every system that should serve the model:

1. Select the system in Overview to open its engine settings.
2. Install llama.cpp, Ollama, or LM Studio if needed. Installation starts the engine; if it is stopped, use its switch to start it.
3. Expand the engine and select Add model.
4. Select a model, select Download, and wait for the download to finish.
5. Load the model if the engine requires a separate load step.

A system can serve a request when it is online, the engine is running, and the requested model is available there. To route requests across several systems, add the same model to the same engine on each system that should serve it. The downloaded model appears in that engine's model list.

llama.cpp example

On RTX Spark, use PAIR's managed llama.cpp installation. PAIR downloads the engine and CUDA runtime separately from the PAIR installer.

1. Under llama.cpp, select Add model.
2. Search for ggml-org/gemma-3-1b-it-GGUF. Typing filters the initial list; press Enter or select the search button to search public Hugging Face repositories if it is not listed.
3. Select the Q4_K_M entry and select Download.
4. Wait until ggml-org/gemma-3-1b-it-GGUF:Q4_K_M appears in the engine's model list. Use Load to load it before testing, or allow the first request to load it.

This small model is an example; you can choose another compatible GGUF model that fits the system's memory. PAIR's llama.cpp catalog offers verified Q4_K_M entries from public repositories. Model IDs include both the repository and quantization: owner/repository:quantization. Use the exact ID when sending a request; an Ollama tag such as gemma3:1b is a different identifier.

Use Eject to release a loaded model's memory while keeping its download. Managed llama.cpp models also sleep after five minutes without inference work; the next request wakes the model. Some GPU memory can remain allocated by the engine.

Copy the local endpoint

Applications connect to PAIR on the system where they run. They do not connect directly to an engine or to the system that will run the request.

1. Select Endpoints in PAIR's top toolbar.
2. In the API endpoints window, find the engine that you prepared.
3. Copy its http://127.0.0.1:<port> URL.

If the window says No engines are running, return to step 4 and start an engine. The endpoint appears when that engine is running on any system in the cluster, including a remote system.

Send a test request

Use the endpoint from step 5 and the exact downloaded model ID from step 4. The examples below use the OpenAI-compatible API to work with all three engines.

Choose a request style

PAIR passes each request to the selected engine without rewriting it.

If you are unsure, use the OpenAI-style request. It works with every supported engine.

Windows PowerShell: llama.cpp example

Open PowerShell on the system running PAIR. Set $pairBaseUrl to the endpoint you copied. With llama.cpp's default proxy port, list the available model IDs:

$pairBaseUrl = "http://127.0.0.1:8080"
(Invoke-RestMethod -Uri "$pairBaseUrl/v1/models").data | Select-Object id

Confirm that the output includes the model you downloaded. Set $pairModel to its exact id, then send the request:

$pairModel = "ggml-org/gemma-3-1b-it-GGUF:Q4_K_M"
$pairRequest = @{
    model = $pairModel
    messages = @(@{
        role = "user"
        content = "Tell me a short story about a dog who learns to skateboard."
    })
    stream = $false
} | ConvertTo-Json -Depth 5
$pairResponse = Invoke-RestMethod -Uri "$pairBaseUrl/v1/chat/completions" -Method Post -ContentType "application/json" -Body $pairRequest
$pairResponse.choices[0].message.content

The response prints the model's answer. To use Ollama or LM Studio in PowerShell, use that engine's endpoint and an exact model ID returned by its /v1/models endpoint.

Linux or macOS: OpenAI-style request

In a terminal, replace <PAIR_BASE_URL> with the endpoint from step 5 and list its models:

curl <PAIR_BASE_URL>/v1/models

Copy the exact id from the response. Replace <MODEL_NAME> with that ID and <PAIR_BASE_URL> with the same endpoint in the request below:

curl <PAIR_BASE_URL>/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<MODEL_NAME>",
    "messages": [
      {
        "role": "user",
        "content": "Tell me a short story about a dog who learns to skateboard."
      }
    ]
  }'

Linux or macOS: Ollama-style request

Use this request only with an Ollama endpoint. The -N option shows the response as it streams.

curl -N <PAIR_BASE_URL>/api/chat \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<MODEL_NAME>",
    "messages": [
      {
        "role": "user",
        "content": "Tell me a short story about a dog who learns to skateboard."
      }
    ]
  }'

Open Overview, select the Jobs filter, and read Ran on or Running on on the job card. A response and a job card show that PAIR routed the request. Send several independent requests to observe routing across systems that have the same engine and exact model ID. One request runs on one system.

Connect an application

Set the application's base URL to the endpoint from step 5. Select a model that you added in step 4.

For an OpenAI-compatible SDK or application that appends the API path, include /v1 in the base URL. For example, llama.cpp uses http://127.0.0.1:8080/v1 with the default proxy port. A direct HTTP request uses the complete path, such as http://127.0.0.1:8080/v1/chat/completions. An Ollama-native client uses the host URL, such as http://127.0.0.1:11434.

PAIR accepts requests only from the system where it is running. The proxy uses plaintext HTTP on loopback. A network request to an address such as http://some-node:11434 returns 403.

Install PAIR on the system where you use the application. Join that system to the cluster, then use its local endpoint. The system does not need a GPU or an engine when another cluster system can serve the request.

PAIR does not provide a network-reachable inference endpoint. Configuring an engine to listen on the network is outside PAIR and creates security exposure that you must manage separately.

Port reference

PAIR's proxy uses the port that an engine normally uses. Engines run behind the proxy on separate server ports.

If OLLAMA_HOST names a different local loopback address, PAIR also serves that address when the port is free. PAIR does not use a remote or HTTPS OLLAMA_HOST value.

PAIR uses these ports to communicate between cluster systems:

The Windows installer adds the firewall rules. If a Linux firewall is restrictive, allow these ports between trusted cluster systems.

Next steps

Change ports or engine launch options

1. Open Overview and expand Engine settings on the local system's card.
2. Expand the engine's Settings.
3. Edit Proxy port, Server port, or the engine arguments, then select Apply. Confirm the restart if PAIR requests it.

PAIR applies the change as one operation and restores the new values when it next starts. Supported paired systems can also be edited from another cluster member. If you change a proxy port, update the application's base URL. Endpoints always shows the current URL.

Keep managed llama.cpp in router mode. Do not add -m or -hf model-selection arguments; download and load models with PAIR's model controls instead. See PAIR engine settings for supported argument notation and settings.

For browser applications, configure only the required CORS origins on the system running the engine. Managed llama.cpp starts without cross-origin browser permission, and PAIR follows the engine's policy. Native clients such as PowerShell do not need a CORS setting.

If another application uses a port that PAIR does not manage, choose a different port in PAIR or stop the other application. Then restart the service from Settings → Service.

Use an engine's command line

PAIR-installed engines are normal installations, but their binaries are not in PATH and their server ports differ from the usual defaults.

PAIR-managed Ollama locations:

On Linux, the table shows the default location. If XDG_CONFIG_HOME is set, PAIR uses it instead of ~/.config. The command below handles either location:

pair_ollama_dir="${XDG_CONFIG_HOME:-$HOME/.config}/Nvidia Corporation/Personal AI Router/engine-bin/ollama"
LD_LIBRARY_PATH="$pair_ollama_dir/lib/ollama" OLLAMA_HOST=127.0.0.1:11435 "$pair_ollama_dir/bin/ollama" list

Windows PowerShell example:

$ollama = "$env:LOCALAPPDATA\Nvidia Corporation\Personal AI Router\engine-bin\ollama\ollama.exe"
$env:OLLAMA_HOST = "127.0.0.1:11435"
& $ollama list

LM Studio installs to its standard location:

~/.lmstudio/bin/lms status

On Windows, the executable is %USERPROFILE%\.lmstudio\bin\lms.exe.

Set OLLAMA_HOST to the engine's Server value when you want to inspect the local engine. Otherwise, the CLI uses proxy port 11434 and returns the cluster-wide view. The proxy shows what the cluster can serve. The server port shows what is on that system.