---
title: "Chat with LLMs Using Open WebUI and Ollama — Open WebUI Remotely"
canonical: "https://build.nvidia.com/playbooks/open-webui/sync.md"
---

> [!TIP]
> These steps assume NVIDIA Sync is installed and connected to your hardware platform. If you still need to set that up, complete NVIDIA Sync remote-access setup for your hardware platform first. See also the [NVIDIA Sync documentation](https://docs.nvidia.com/sync/latest/direct-connections.html#nvidia-sync-direct-connections).

# Step 1. Use NVIDIA Sync to connect and open a terminal

From your laptop:

- Open NVIDIA Sync with the desktop icon or from the system tray or taskbar.
- Select your hardware platform from the device dropdown.
- Select **Connect**.
- After the connection is established, select Terminal to open a terminal on the hardware platform.

# Step 2. Configure Docker permissions

You must first make sure that your user account can run Docker commands on the hardware platform without sudo.

To test that, in the terminal run:

```bash
docker ps > /dev/null
```

**Success:** If the command returns a blank, then skip ahead to Step 3.

Otherwise, you will see a permission denied error, which means you still need to remove the sudo requirement.
To do that, add your user to the docker group with the commands below.

```bash
sudo usermod -aG docker $USER
newgrp docker
```

Then verify the change is set by testing Docker access again with the command:

```bash
docker ps > /dev/null
```

# Step 3. Download the Open WebUI container image

Pull the container image onto your hardware platform with the command:

```bash
docker pull ghcr.io/open-webui/open-webui:ollama
```

Wait for the image to download, then go to Step 4.

# Step 4. Add Open WebUI as a custom application through NVIDIA Sync

A custom application lets NVIDIA Sync start Open WebUI and automatically forward its port.

In the NVIDIA Sync device window:

- Select **Add New** in the **Custom** section.
- Fill out the form with these values:
- **Name:** Open WebUI
- **Port:** 12000
- **Auto open in browser at the following path:** Check this checkbox

- Then, copy and paste the entire script below into the **Launch Script** field

```bash
#!/usr/bin/env bash
set -euo pipefail

NAME="open-webui"
IMAGE="ghcr.io/open-webui/open-webui:ollama"

cleanup() {
echo "Signal received; stopping ${NAME}..."
docker stop "${NAME}" >/dev/null 2>&1 || true
exit 0
}
trap cleanup INT TERM HUP QUIT EXIT

# Ensure Docker CLI and daemon are available
if ! docker info >/dev/null 2>&1; then
echo "Error: Docker daemon not reachable." >&2
exit 1
fi

# Already running?
if [ -n "$(docker ps -q --filter "name=^${NAME}$" --filter "status=running")" ]; then
echo "Container ${NAME} is already running."
else
# Exists but stopped? Start it.
if [ -n "$(docker ps -aq --filter "name=^${NAME}$")" ]; then
echo "Starting existing container ${NAME}..."
docker start "${NAME}" >/dev/null
else
# Not present: create and start it.
echo "Creating and starting ${NAME}..."
docker run -d -p 12000:8080 --gpus=all \
-v open-webui:/app/backend/data \
-v open-webui-ollama:/root/.ollama \
--name "${NAME}" "${IMAGE}" >/dev/null
fi
fi

echo "Running. Press Ctrl+C to stop ${NAME}."
# Keep the script alive until a signal arrives
while :; do sleep 86400; done
```

- Finally, click the **Add** button to finish the configuration.

# Step 5. Launch Open WebUI and create an administrator account

Once the app is configured, you can launch it from NVIDIA Sync and connect to it with your browser.

In the NVIDIA Sync application window for your hardware platform, select **Open WebUI** in the **Custom** section.

The application should open in your web browser at the URL `http://localhost:12000`.

If it does not, open your web browser and go to `http://localhost:12000`.

Open WebUI uses a local administrator account to control access. The account credentials are stored locally on your hardware platform.

When the app opens in your browser, create your admin account as follows:

- Select **Get Started** at the bottom of the screen.
- Complete the admin account creation with easily remembered details.
- Select **Create Admin Account** to complete.

# Step 6. Select a model to download

> [!TIP]
> The Open WebUI container doesn't come with a model, so you must download one before chat will work.
> Open WebUI downloads selected models from the Ollama [registry](https://ollama.com/search).

Do the following in the Open WebUI application:

- Click **Select a model** in the top left corner of the Open WebUI interface.
- Type `gpt-oss:20b` in the search field.
- Click the **Pull "gpt-oss:20b" from Ollama.com** button that appears.
- Wait for the model to fully download. You can monitor progress in the interface.

Alternatively, you can enter `qwen3.6:latest` instead of `gpt-oss:20b`.

After the download completes, the model appears in the **Select a model** menu.

# Step 7. Load the model and submit a query

> [!TIP]
> Selecting an available model loads it onto the GPU, which can take up to 30 seconds, depending on the model size.
> This can delay server response to your initial query.

- Select the model from the **Select a model** menu in the top-left corner.
- In the chat box, enter a prompt such as `Write me a haiku about GPUs` and press Enter.

# Step 8. Stop Open WebUI with NVIDIA Sync

When you finish your session, you can stop the Open WebUI container from the NVIDIA Sync application window.

- Click on the NVIDIA Sync icon in your system tray or taskbar to open the main application window.
- Under the **Custom** section, click the `x` icon on the right of the **Open WebUI** entry.
- This closes the tunnel and stops the Open WebUI Docker container.

# Step 9. Next steps

You can follow up with other playbooks or use different models.

- For agentic workloads, see the **Agent-ready Models** tab for the recommended model on your hardware platform.
- [Find and compare models from the Ollama model registry](https://ollama.com/library).
- Monitor GPU and system usage during inference (for example, via tools available through NVIDIA Sync).

# Step 10. Cleanup

Use these steps when you want to remove Open WebUI from your hardware platform. Cleanup is optional rollback—not required to finish the playbook.

> [!WARNING]
> These commands permanently delete all Open WebUI data and downloaded models on the hardware platform.

1. Stop the Open WebUI application in the NVIDIA Sync device window (this will also stop the container).

2. Open a terminal on the hardware platform using the Terminal App in the NVIDIA Sync device window.

3. Remove the container with the command:

```bash
docker rm open-webui
```

4. Remove the downloaded image with the command:

```bash
docker rmi ghcr.io/open-webui/open-webui:ollama
```

5. Remove the persistent data volumes with the command:

```bash
docker volume rm open-webui open-webui-ollama
```

6. Remove the custom application from NVIDIA Sync by opening the device window and deleting the **Open WebUI** entry from the **Custom** section.