Set Up RTX Spark as Your Always-On AI Hub

Use RTX Spark as your inference server

Your agent stays on your laptop, close to your files and apps, and RTX Spark runs the AI model that powers it. Other PCs and apps in your home can share the same model. If you want the agent itself to run on RTX Spark, use the Agents tab instead.

Before you start: complete the Remote Access tab, and install your agent app on your laptop.

Choose a route

Start with PAIR if your laptop stays at home. Your agent sends requests to PAIR on your laptop, and PAIR routes each request to a paired computer that has the model. With the model only on RTX Spark, every request runs there. Choose Tailscale if you also want to use RTX Spark away from home, or if a device cannot run PAIR. You can set up both routes.

Set up your route

Route A: NVIDIA PAIR

1. Follow Steps 1 to 5 of Set Up the PAIR App, then return here. In those steps, you install PAIR on RTX Spark and on your laptop, pair the two, add a model on RTX Spark, and copy the llama.cpp endpoint from PAIR on your laptop.
2. Add /v1 to the end of the endpoint you copied. This is your server URL. It looks like http://127.0.0.1:8080/v1. The address is on your laptop because your agent talks to PAIR there, and PAIR passes each request on to RTX Spark.

Route B: Tailscale

1. On RTX Spark, follow Option 1: llama.cpp in Run LLMs on Your NVIDIA Device, from Download through Test, then return here. Leave the server on 127.0.0.1:9931.

2. Set up Tailscale on RTX Spark and on your laptop (Remote Access tab, Step 4).

3. In PowerShell on RTX Spark, share the server with your Tailscale devices:

   Powershell

   CopiedCopy

   tailscale serve --bg --https=443 http://127.0.0.1:9931
   

   If Tailscale prompts you to enable HTTPS, follow the link it shows, then run the command again.

4. Check that the server is shared, and copy the HTTPS address:

   Powershell

   CopiedCopy

   tailscale serve status
   

5. Add /v1 to the end of that address. This is your server URL. It looks like https://rtx-spark.your-tailnet.ts.net/v1.

Only devices on your Tailscale network can reach this address, and Tailscale keeps sharing it after a restart. To stop sharing, run tailscale serve reset. For more detail, see Tailscale Serve and Tailscale access controls.

Connect your agent and apps

1. On your laptop, open your agent’s model or provider settings and add a custom endpoint.
2. Enter these settings:
   • Base URL: your server URL, including /v1.
   • Model: the model that RTX Spark serves.
   • API key: any placeholder text, if the app requires a key.
3. Save the endpoint and select the model.

Any other app that works with an OpenAI-compatible endpoint, such as a chat app or a coding assistant, can use the same settings.

Check your setup

1. In PowerShell on your laptop, list the models that RTX Spark serves. Use your own server URL:

   Powershell

   CopiedCopy

   curl.exe https://rtx-spark.your-tailnet.ts.net/v1/models
   

   The response should include your model.

2. Send your agent a message. It should answer.

3. For the Tailscale route, connect your laptop to a phone hotspot and repeat both checks to confirm that it works away from home.

After RTX Spark restarts, sign in through remote desktop and check that the model server is running. With PAIR, open PAIR and check that the engine is running. With llama.cpp, run the server command again.