--- title: "Run Healthcare Agents with Local Inference — Instructions" canonical: "https://build.nvidia.com/playbooks/healthcare-agent/instructions.md" --- > [!IMPORTANT] > This playbook requires Docker (with NVIDIA runtime), Node.js v22, and OpenShell CLI >= 0.0.111 — both the `openshell` CLI **and** the `openshell-gateway` daemon, which the official OpenShell installer provides together (Step 1). You do **not** need the NemoClaw playbook: this playbook only uses the OpenShell binaries, and running the full NemoClaw stack first leaves services (host Ollama on 11434, `nemoclaw-vllm` on 8000, `openclaw-gateway` on 18789) that collide with this playbook — Step 1 and Step 3 explain how to clear them if you already ran it. Ollama runs as a Docker container here (host Ollama is not required and will conflict with port 11434, see Step 3). > [!NOTE] > Steps 1–3 are prerequisites. Steps 4–5 configure infrastructure and deploy the agent. Steps 6–9 are the demo. Steps 10–11 are cleanup and next steps. # Step 1. Verify your environment Confirm your hardware platform has the required software and free disk space: ```bash # OpenShell installs to ~/.local/bin, which is not on the default # non-interactive PATH. Add it before running anything below. echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc export PATH="$HOME/.local/bin:$PATH" nvidia-smi docker info --format '{{.ServerVersion}}' node --version ollama --version 2>/dev/null || echo "ollama not installed (OK — Docker provides it)" openshell --version df -h / ``` Expected: a large-memory Blackwell GPU on the supported hardware platform, Docker >= 23.0.1, **Node.js v22.x**, OpenShell >= 0.0.111, and **at least 200 GB free** on `/` (86 GB model + Docker images + working space). > [!WARNING] > If `openshell --version` says `command not found` but the binary exists at `~/.local/bin/openshell`, it just isn't on PATH. Run the `export PATH=...` line above and re-source `~/.bashrc`. Without this, every `openshell` and `make` command in later steps fails. **If OpenShell is not installed at all**, install it with the official installer — this installs both the `openshell` CLI and the `openshell-gateway` daemon, which is everything this playbook needs (no NemoClaw required): ```bash curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/main/install.sh | sh source ~/.bashrc # put ~/.local/bin on PATH openshell --version # should show >= 0.0.111 ``` > [!TIP] > `make prereq` (run from `~/clinical-intelligence` after Step 2) bundles all of the checks below — Docker, Node version, OpenShell, disk space, GPU, port 11434, and NGC auth — into one command. **If `node --version` reports v18.x, older, or `command not found`**, install Node.js v22 before continuing: ```bash # Download the NodeSource setup script first, then run it with sudo. # Running it inline with `| sudo bash` does not work — the sudo context # needs to own the entire script execution. curl -fsSL https://deb.nodesource.com/setup_22.x -o /tmp/nodesource_setup.sh sudo bash /tmp/nodesource_setup.sh sudo apt-get install -y nodejs node --version # should now show v22.x ``` **Host Ollama must not be on port 11434.** This playbook runs Ollama in a Docker container that binds 11434 on the host. Run **both** of these: ```bash # Always run — succeeds silently if no service exists, stops the host # Ollama daemon if NemoClaw or another playbook left one running. sudo systemctl stop ollama 2>/dev/null || true sudo systemctl disable ollama 2>/dev/null || true # Verify nothing else owns 11434. ss -tlnp 2>/dev/null | grep 11434 || echo 'port 11434 free' ``` Expected: `port 11434 free`. If the line still shows a listener, something else (an old `ollama serve`, another container, etc.) owns the port — stop it, or change `OLLAMA_PORT` in `.env` (Step 2) to a free port such as `11435`. `make setup` sources `.env` and configures the sandbox provider against the override. **Stale OpenShell gateway?** Do not kill the package-managed gateway. Step 4 uses its systemd user service and replaces a stale CLI registration with the authenticated HTTPS endpoint. If a legacy `--disable-tls` process is still running, the setup helper detects it and asks you to stop that one process. ```bash systemctl --user status openshell-gateway.service --no-pager || true openshell gateway list ``` **Previously ran the NemoClaw playbook?** NemoClaw installs `openclaw-gateway.service` as a systemd user service that binds port 18789. If it is still running, `make setup` fails with "Port 18789 is already in use". Stop and disable it before proceeding — `make setup` will also do this automatically, but stopping it here avoids a confusing error: ```bash systemctl --user stop openclaw-gateway.service 2>/dev/null || true systemctl --user disable openclaw-gateway.service 2>/dev/null || true # Verify the port is free ss -tlnp | grep 18789 || echo 'port 18789 free' ``` # Step 2. Copy the assets and configure The playbook assets include the Docker Compose file, agent definitions, skill files, and setup scripts. Locate the `assets/` directory shipped with this playbook and copy it to `~/clinical-intelligence`: ```bash # Find the playbook directory first. Try the common locations: PLAYBOOK_DIR="" for d in ~/client-hardware-playbooks/nvidia/playbook-healthcare-agent \ /opt/client-hardware-playbooks/nvidia/playbook-healthcare-agent \ /usr/local/share/client-hardware-playbooks/nvidia/playbook-healthcare-agent; do if [ -d "$d/assets" ]; then PLAYBOOK_DIR="$d"; break; fi done if [ -z "$PLAYBOOK_DIR" ]; then echo "ERROR: assets/ not found. Locate it manually:" echo " find / -type d -path '*playbook-healthcare-agent/assets' 2>/dev/null" echo "Then re-run with PLAYBOOK_DIR set." else ( set -e INSTALL_DIR="$HOME/clinical-intelligence" mkdir -p "$INSTALL_DIR" # Copy the contents (including dotfiles) into a stable install root. Using # assets/. avoids creating INSTALL_DIR/assets when this step is re-run. cp -a "$PLAYBOOK_DIR/assets/." "$INSTALL_DIR/" cd "$INSTALL_DIR" # Preserve an existing configured NGC key on repeat runs. if [ ! -f .env ]; then cp .env.example .env fi nano .env # Set: NGC_API_KEY=nvapi-... ) fi ``` > [!IMPORTANT] > If the catalog tarball did not include `assets/`, pull the playbook directly from the repo: > ```bash > git clone https://github.com/NVIDIA/dgx-spark-playbooks ~/client-hardware-playbooks > # Then re-run the Step 2 block above. It will find this clone, copy the > # asset contents without nesting assets/, and preserve an existing .env. > ``` The NGC API key is required to **download** the OpenFold3 NIM image from `nvcr.io` and to **run** it at runtime. Get one for free at [ngc.nvidia.com](https://ngc.nvidia.com/setup/api-key). Authenticate Docker against NGC so the OpenFold3 image pull succeeds (without this you get a raw HTML 401 from nginx): ```bash make ngc-login # or, equivalent manual command: # echo "$NGC_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin ``` # Step 3. Start Nemotron 3 Super and OpenFold3 Ollama serves Nemotron 3 Super for LLM inference. OpenFold3 NIM handles protein structure prediction. Both run as Docker containers with GPU access. If you are connected via SSH, start a `tmux` or `screen` session first so the download survives a disconnection: ```bash tmux new -s clinical ``` Start the services: ```bash docker compose up -d ollama openfold3 ``` > [!IMPORTANT] > **Previously ran the NemoClaw playbook?** NemoClaw's `nemoclaw-vllm` inference container binds host port **8000** — the same port OpenFold3 needs. If OpenFold3 fails to start with a "port is already allocated" / "address already in use" error on 8000, either stop the NemoClaw container: > ```bash > docker stop nemoclaw-vllm && docker rm nemoclaw-vllm > ``` > or give OpenFold3 a different host port by setting `OPENFOLD_PORT=8001` in `.env` (Step 2). `docker-compose.yml`, `make status`, and the test suite all honor the override. Check what holds 8000 with `ss -tlnp | grep :8000`. > [!NOTE] > First run downloads Nemotron 3 Super (~86 GB). This takes 15–25 minutes on a fast connection, up to an hour on slower links. Monitor with `docker compose logs -f ollama`. If interrupted, re-run — it resumes where it left off. > [!TIP] > **Multi-GPU hardware platforms:** docker-compose pins both Ollama and OpenFold3 to GPU 0 by default (set `LLM_GPU` and `OPENFOLD_GPU` in `.env` to override). On a dual-GPU hardware platform, set `LLM_GPU` and `OPENFOLD_GPU` to the index of the **large-memory GPU** because Nemotron-3-Super (~94 GB resident) does not fit safely on a ~98 GB discrete GPU. Find the large-memory GPU index with: `nvidia-smi --query-gpu=index,name,memory.total --format=csv,noheader` Pull the model into Ollama: ```bash docker compose up model-pull ``` Wait for all services to report healthy: ```bash make status ``` Expected: ``` Ollama (port 11434): ✓ healthy OpenFold3 (port 8000): ✓ healthy ``` OpenFold3 takes ~3 minutes to load model weights on startup. If it shows "down (may still be loading)", wait and check again. > [!TIP] > `make up` runs all of the above (container start + model pull) in one command but does not block on health. Run `make status` separately to verify services are healthy. # Step 4. Start the OpenShell gateway OpenShell's Linux package owns the gateway lifecycle through `openshell-gateway.service`. The checked-in TOML pins the Docker driver while keeping TLS and local mTLS authentication enabled. The helper also tests Docker from the systemd user manager's credential context; this catches a login that has the `docker` group while an older user manager does not. ```bash bash scripts/ensure_openshell_gateway.sh # Remote/headless Linux: keep the package user service alive after SSH logout. sudo loginctl enable-linger "$USER" loginctl show-user "$USER" -p Linger --value # expected: yes ``` Under the hood this uses the supported lifecycle and registration: ```bash systemctl --user enable openshell-gateway.service systemctl --user restart openshell-gateway.service openshell gateway add https://127.0.0.1:17670 --local --name openshell openshell gateway select openshell openshell -g openshell status openshell -g openshell gateway info --output json ``` Expected: `Status: Connected`, `Authentication: Authenticated`, and a healthy Docker compute driver. `Connected` alone is not sufficient because the health RPC is public. For service failures, run `journalctl --user -u openshell-gateway.service --no-pager -n 50`. If the helper says the systemd user manager cannot access Docker, fully end all login sessions and log back in—or reboot—after Docker-group membership was assigned, then rerun. If linger is already enabled, use a maintenance reboot: logout does not recreate a lingering manager. Restarting only the gateway cannot refresh the manager's inherited groups. > [!NOTE] > Step 4 configures OpenShell infrastructure (gateway). Step 5 deploys the healthcare agent into this infrastructure. # Step 5. Deploy the healthcare agent `make setup` automates six operations: 1. Creates the `clinical-sandbox` OpenShell sandbox with the network policy from `sandbox-policy.yaml` 2. Creates the inference provider and configures routing to Ollama (previously a manual step) 3. Installs Python packages (requests, pandas, matplotlib) inside the sandbox 4. Copies agent definitions from `agents/` and skill files from `skills/` into the sandbox workspace 5. Deploys the OpenClaw configuration (`openclaw.json`, `IDENTITY.md`) and registers specialist agents 6. Starts the OpenClaw gateway and runs a smoke test ```bash make setup ``` When complete, you will see `=== Setup Complete ===`. If setup fails, re-run `make setup` — it recreates the sandbox from scratch, so all config is fresh. Verify the sandbox config matches the repo: ```bash make check ``` All checks should pass. If any skills or config files are stale, `make check` tells you what to fix. Run the quick test suite: ```bash make test ``` > [!TIP] > `make setup` runs `bash scripts/setup_sandbox.sh` (loopback bind — use the SSH tunnel from Step 6 for remote browsers). For direct local-browser access on the hardware platform itself, run `make setup-local` instead, which calls `bash scripts/setup_sandbox.sh --local`. See `RUNBOOK.md` in the assets for the manual step-by-step equivalent. # Step 6. Open the dashboard **Remote access** (run this from your machine, not the hardware platform). This forwards port 18789 so you can open the dashboard in a local browser: ```bash ssh -f -N -L 18789:localhost:18789 your-user@your-hardware-platform-ip ``` **Cursor / VS Code:** Open the **Ports** tab in the bottom panel, click **Forward a Port**, enter **18789**. Then open `http://localhost:18789/` in your browser. **Local** (keyboard and monitor on the hardware platform): Open `http://localhost:18789/`. You should see the OpenClaw dashboard with **Health: OK**. Click **Chat**. Setup is done. The next steps are the demo. # Step 7. Run healthcare queries All queries execute inside the OpenShell sandbox — only whitelisted endpoints are reachable. You will verify this in Step 9. Paste the first query. The first response takes 30–60 seconds while the 120B model loads into GPU memory — this is normal and only happens once per session. ``` Find all diabetic patients and get their latest HbA1c. Generate a histogram with a red dashed line at 9%. Use dark background with green bars. ``` The agent reads its skill files (`fhir-basics`, `clinical-knowledge`, `analysis-methods`), imports the FHIR helpers library, writes a Python script, queries the FHIR server, and generates the chart — all inside the sandbox. You should see ~48 diabetic patients and an A1c histogram with a 9% threshold line. The agent includes a clickable link to the chart in its response. You can also browse all visualizations at `http://localhost:18789/__openclaw__/canvas/`. Stay in the same session for follow-ups: ``` Which of those diabetic patients also have hypertension? For the overlap, get their eGFR. Flag anyone with eGFR below 60 as kidney disease risk. ``` You should see ~24 with both conditions, ~12 flagged as kidney disease risk. ``` Of those kidney disease risk patients, which ones are not on an ACE inhibitor or ARB? ``` You should see ~12 patients missing guideline-recommended therapy (100% care gap in the synthetic data). # Step 8. Visualize a drug target ``` Show me the 3D protein structure of atorvastatin bound to its target ``` The molecular agent looks up atorvastatin's target protein (HMG-CoA reductase), fetches the drug's SMILES from PubChem, sends the protein sequence to OpenFold3 for structure prediction, and generates an interactive 3D viewer with confidence scores (pLDDT, pTM, ipTM). You should see an interactive 3D viewer with the protein ribbon structure and atorvastatin ligand. The agent includes a clickable link. Confidence scores appear in the viewer header — pLDDT > 70 indicates a good prediction. # Step 9. Understand sandbox isolation The sandbox policy (`sandbox-policy.yaml`) enforces implicit-deny networking at Layer 7. Every outbound connection is blocked unless an explicit rule allows it. The policy also restricts HTTP methods — FHIR and PubChem are limited to read-only (GET/HEAD), OpenFold3 accepts only POST to specific prediction paths. Confirm that unauthorized endpoints are blocked. From inside the sandbox (connect with `openshell sandbox connect clinical-sandbox`), run: ```bash curl --max-time 5 https://google.com ``` Expected: connection refused or `CONNECT tunnel failed, response 403`. The allowed endpoints are: | Endpoint | Purpose | Allowed methods | |----------|---------|----------------| | `https://inference.local` (port 443 only) | LLM calls to Ollama | All (OpenAI protocol) | | `r4.smarthealthit.org` | FHIR patient data | GET, HEAD | | `pubchem.ncbi.nlm.nih.gov` | Drug SMILES lookup | GET, HEAD | | OpenFold3 NIM (Docker bridge) | Structure prediction | POST `/biology/openfold/**`, GET `/v1/health/*` | | CDN (jquery, 3dmol, unpkg) | JavaScript for 3D viewers | GET | Everything else is denied. Additional rules for GitHub, npm, and PyPI are included in `sandbox-policy.yaml` for build dependencies during sandbox setup — these are setup-only and not used at runtime. > [!NOTE] > `inference.local` is HTTPS-only. Plain `http://inference.local/...` returns `policy_denied` because the OpenShell L7 proxy enforces the TLS-terminated CONNECT path. All skills, helper scripts, and `_sandbox` curls in this repo use `https://inference.local` with `-k` (the proxy presents a self-signed cert). Patient data flows from FHIR → sandbox Python execution. It never passes through the LLM, OpenFold3, or PubChem. Inspect a skill file to see the editable clinical knowledge: ```bash head -30 skills/clinical-knowledge/SKILL.md ``` Skill files are Markdown. Edit a threshold or drug classification — it takes effect on the next query, no retraining. Try changing the HbA1c threshold from `9.0%` to `7.0%` and re-running the diabetes query to see the difference. # Step 10. Cleanup > [!WARNING] > This removes the sandbox and stops all services. ```bash openshell sandbox delete clinical-sandbox make down ``` The package-managed OpenShell gateway is shared infrastructure and remains running. Stop it explicitly with `systemctl --user stop openshell-gateway` only when you intend to stop OpenShell for every local playbook. To also remove downloaded models and volumes: ```bash make clean ``` # Step 11. Next steps 1. **Edit a skill** — change a lab reference range in `skills/clinical-knowledge/SKILL.md` and re-run the same prompt to see the effect. 2. **Add an agent** — create a `.md` in `agents/`, register in `openclaw.json`, redeploy with `make setup`. 3. **Connect a real FHIR server** — replace the test server URL, add OAuth2 authentication, and update `sandbox-policy.yaml` with explicit path-level rules for the FHIR resources the agent needs. 4. **Swap models** — try `qwen2.5:72b` or another Ollama model by editing `.env` and `openclaw.json`. 5. **Verify after changes** — run `make check` after any config or skill file change to catch stale sandbox copies. 6. **Full agent validation** — `make test-full` runs all test levels including end-to-end agent queries (~20 min). 7. **Monitor GPU** — `nvidia-smi` shows memory allocation across Nemotron 3 Super and OpenFold3.