Install and Use NVIDIA PAIR with llama.cpp

Start the terminal interface

Use the PAIR terminal interface on a system without a desktop environment or over SSH. Quit the desktop application first, including its tray or menu-bar instance. Run one PAIR interface per system: the desktop application and terminal interface each start services that use the same ports, engines, and settings.

These instructions target PAIR 1.0.0 and its five-tab terminal interface. Install the matching 1.0.0 platform package from the NVIDIA PAIR download page or PAIR releases page. The app setup tab lists installer choices by OS and architecture. On Windows, launching the installed desktop application creates the nvpair command wrapper. On macOS it does so when /usr/local/bin is writable. Quit the application before using that command and open a new terminal after the wrapper is created. On Windows, use native PowerShell. The Debian package creates the wrapper during installation.

nvpair --version
nvpair

For a system where you will not launch the desktop application, download and extract the 1.0.0 services and terminal-interface archive for your platform and architecture instead. Keep all its binaries together. From the directory containing them, run:

Windows PowerShell:

.\nvpair-tui.exe --version
.\nvpair-tui.exe

Linux or macOS:

./nvpair-tui --version
./nvpair-tui

Confirm the downloaded package or archive belongs to PAIR 1.0.0. The --version command prints the terminal-interface component version, which can differ from the PAIR release version. The terminal interface starts its PAIR service tree; no other PAIR process needs to be running first. The header shows service ready and a version when it connects. If it stays on starting service..., inspect Logs (tab 5).

Pair this system

One system can serve local inference. To route requests across systems, start PAIR on each system and pair them on a trusted local network. The six-digit PIN is a temporary setup code. Inviting the first system forms the cluster automatically.

To pair with a discovered system:

1. Open Nodes (tab 1).
2. Select the other system with j / k or the arrow keys.
3. Press p.
4. Give the displayed PIN to the person operating the other system.

If discovery has not found the system, press n on Nodes, enter its hostname or IP address, and press enter. Use host:port when the system does not use the default pairing port.

On the invited system, open Nodes, press a when the pairing request appears, enter the PIN, and press enter. Press d to decline. When pairing succeeds, the peer's CLUSTER column reads Member.

Press c to cancel an invitation you sent. A wrong or expired PIN requires a new invitation. Repeat pairing to add more systems; each system can belong to one cluster at a time.

Start an engine and add a model

In Nodes, select your own system and press enter to open its hardware, Engines, and Models. Move between the two panes with h / l or ← / →. Within a pane, select a row with j / k or ↓ / ↑.

For a llama.cpp example:

1. In Engines, select llama.cpp and press i to install it.
2. Press s to start it. Wait for RUNNING and HEALTHY to read yes.
3. Keep llama.cpp selected and move to Models.
4. Press n, enter ggml-org/gemma-3-1b-it-GGUF:Q4_K_M, and press enter.
5. Wait for the download to finish and the model to appear in the model list.
6. Select that model and press enter to load it. Confirm LOADED reads yes.

Select the intended running engine before downloading: the download goes to the highlighted running engine. llama.cpp uses a Hugging Face GGUF repository and quantization as its model ID. Keep the exact ID, including :Q4_K_M, for model operations and inference requests.

You can browse instead of typing a name. In Models, press p to open the selected engine's download catalog. Select a model and press enter to download it. For llama.cpp, / opens a Hugging Face search; enter a query and press enter. Press c to return to the initial popular-model list, o to change the sort order, and esc to leave the catalog. Other engines use / to filter their catalog by name, family, or parameter size.

The same node-detail screen can install, start, and stop engines and manage models on paired peers. Restart and uninstall are available on the local system only. Destructive operations require y to confirm; any other key cancels.

A system can serve a request when it is online, its compatible engine is running, and the requested model is available there. Prepare the same exact model on multiple systems when you want any of them to serve it. Press esc to return to the Nodes list and check each system's model inventory.

Check the service and endpoint

Open Service (tab 3) and confirm the service is connected. Worker health is best-effort: ok means no crash was reported, DOWN means a worker reported a crash, and ? means PAIR is still checking or the service is not answering. The error-reporting worker cannot report its own crash. Use Errors and Logs to investigate a failure.

Open Jobs (tab 2). Its top line shows each engine's current local proxy port. Use the port for the engine you prepared to form http://127.0.0.1:<port>. This is the client endpoint; the engine's own port is a separate value.

Read the actual port in Jobs, because it can differ from the default. Keep the terminal interface running and open another terminal on the same system to check the endpoint. The examples below use llama.cpp's default; replace the URL if Jobs shows another port.

Windows PowerShell:

$pairBaseUrl = 'http://127.0.0.1:8080'
Invoke-RestMethod -Uri "$pairBaseUrl/v1/models" | Select-Object -ExpandProperty data

Bash:

PAIR_BASE_URL=http://127.0.0.1:8080
curl "$PAIR_BASE_URL/v1/models"

The response lists the cluster's available model IDs, including the model you prepared. Use an exact returned ID in your client. An OpenAI-compatible client normally uses <PAIR_BASE_URL>/v1 as its base URL and appends API paths itself. All three engines accept OpenAI-style chat requests; Ollama's /api/chat is available only on the Ollama endpoint.

The proxy endpoint accepts requests from its own system only. Run PAIR on the system hosting your client and pair it with the systems serving models. That client system does not need its own engine or GPU.

Check routing activity

Open Jobs and press t to generate up to sixty seconds of synthetic inference traffic through the local endpoints. It needs a running engine with a text-generation model available. The progress line shows requests sent and time remaining. Press t again to stop new requests early; requests already sent can finish.

Jobs appear in the table. FROM identifies the system where a request arrived; RAN ON identifies the system that served it. Press a to include finished jobs. Different systems in those columns demonstrate routing to a peer. PAIR selects a system for each request automatically; one request runs on one system.

The synthetic test shows routing activity without displaying prompts or responses. To verify an answer, send a request from your configured client and confirm both its reply and the serving system in Jobs. To prove remote routing, prepare the requested model on a peer only, send from your local endpoint, and check that RAN ON names that peer.

Next steps

Keep PAIR running after an SSH disconnect

Quitting the terminal interface stops its PAIR services. An SSH disconnect also ends it. On a Linux system reached over SSH, use a terminal multiplexer when PAIR must stay available:

tmux new -s pair
nvpair

If you extracted a services archive, start ./nvpair-tui from its binary directory inside the tmux session instead. Detach with Ctrl-b d and reattach with tmux attach -t pair. GNU Screen also works: start with screen -S pair, detach with Ctrl-a d, and reattach with screen -r pair. Neither tool is included with PAIR.

Move around the terminal interface

Use a terminal at least 40 columns wide and 12 rows high. The tab bar and footer show the current controls. When a text field is open, press enter to submit or esc to cancel before changing tabs.

Use the terminal interface tabs

Inspect errors and logs

Errors shows its active-error count in the tab label. Select an entry to read its operation and suggested action. Press c to clear an error on the system that reported it; a peer's error must be cleared from that peer.

In Logs, scroll with j / k, press / to filter, c to clear the filter, and t to toggle tailing. Press s to save the full log buffer to a timestamped file in your home directory. Change the service log level from Service.

Change ports and startup arguments

On Nodes, open a system's detail screen and select an engine in Engines. Press e to edit its engine port, p to edit its proxy port, or a to edit startup arguments and environment assignments. Clients use the proxy port. When changing it, update your client's base URL and check the resulting port in Jobs on that system.

These settings can also be edited on paired peers when the engine supports configuration; changes to CORS settings must be made on the engine's own system. An engine PAIR found already running can report that its settings are not editable. Startup arguments use literal shell-style quoting, with environment assignments first; variables such as $HOME and %USERPROFILE% are not expanded. Do not include the engine executable or startup subcommand.

Press enter to validate and save. PAIR validates syntax and its managed network settings; other options must be supported by the engine. If applying the change requires an engine restart, confirm with y. Read the reported result rather than assuming the requested port was available.

Command flags and current limits

The terminal interface can notify you of PAIR updates but does not install them. It cannot update an inference engine; use the desktop application for that operation. The Jobs test generates synthetic traffic, while a compatible external client sends your own prompts and displays the replies.