Run Healthcare Agents with Local Inference

Docker and infrastructure

Symptom	Cause	Fix
`make up` hangs on model pull	Nemotron-3-Super is ~86 GB and takes 15–25 min on first download (longer on slow links)	Wait. Check progress with `docker compose logs -f ollama`. If interrupted, re-run — it resumes where it left off.
`OpenFold3: ✗ down` in `make status`	OpenFold3 takes ~3 minutes to load model weights on startup	Wait and re-run `make status`. Check logs with `docker compose logs -f openfold3`.
OpenFold3 crash-loops with `NIMProfileIDNotFound: Profile not found for this model`	An older or cached OpenFold3 NIM image contains a PCI-gated profile manifest that does not recognize the GPU.	Pull and recreate the digest-pinned OpenFold3 1.6.0 service — see "OpenFold3: GPU not recognized" below. Do not patch the manifest.
`failed to bind host port for 0.0.0.0:11434` on `docker compose up ollama`	Host Ollama is already listening on 11434 (common after the NemoClaw playbook)	Stop host Ollama: `sudo systemctl stop ollama && sudo systemctl disable ollama`. Or override in `.env`: `OLLAMA_PORT=11435` — `make setup` and `setup_sandbox.sh` source `.env` and configure the sandbox provider against the new port.
`failed to bind host port for 0.0.0.0:8000` / "address already in use" on `docker compose up openfold3`	NemoClaw's `nemoclaw-vllm` container already holds port 8000 (common after the NemoClaw playbook)	Stop it: `docker stop nemoclaw-vllm && docker rm nemoclaw-vllm`. Or override in `.env`: `OPENFOLD_PORT=8001` — `docker-compose.yml`, `make status`, and the tests honor it. Inspect with `ss -tlnp \| grep :8000`.
`unauthorized: <html><head><title>401 Authorization Required` when pulling `nvcr.io/nim/openfold/openfold3`	Docker is not authenticated against NGC; `NGC_API_KEY` in `.env` is the runtime credential, not the pull credential	Run `make ngc-login` (reads `NGC_API_KEY` from `.env`). Manual equivalent: `echo "$NGC_API_KEY" \| docker login nvcr.io -u '$oauthtoken' --password-stdin`.
OpenFold3 crashes with `device >= 0 && device < num_gpus INTERNAL ASSERT FAILED`	OpenFold3's PyTorch backend rejects multi-GPU containers; `count: all` exposes both GPUs on a dual-GPU hardware platform	`docker-compose.yml` pins to `LLM_GPU`/`OPENFOLD_GPU` (default `0`). On dual-GPU hardware platforms, set both to the large-memory GPU index in `.env` and `docker compose up -d --force-recreate openfold3`.
`NGC_API_KEY not set` error	`.env` file missing or NGC key not configured	Run `cp .env.example .env` and edit to add your NGC API key from ngc.nvidia.com.
`exec format error` when pulling containers	Container architecture mismatch (x86 container on ARM64)	Ensure you're using ARM64-compatible containers. OpenFold3 (v1.3.0+) and Ollama support ARM64. Check with `docker inspect --format '{{.Architecture}}' <image>`.
Sandbox policy validation fails on startup	`landlock: hard_requirement` aborts if filesystem paths can't be enforced	Check that all paths in `sandbox-policy.yaml` exist on the system. If running on a non-standard OS image, try `compatibility: best_effort` temporarily to diagnose.
`node: command not found` or OpenShell rejects the Node version	Node.js missing, or an older image ships v18; OpenShell/OpenClaw need v22+	Download the setup script first, then run it — piping straight into `sudo bash` fails: `curl -fsSL https://deb.nodesource.com/setup_22.x -o /tmp/nodesource_setup.sh && sudo bash /tmp/nodesource_setup.sh && sudo apt-get install -y nodejs`. `make prereq` validates the version automatically.

Gateway and sandbox

Symptom	Cause	Fix
`openshell status` shows `Connected` plus `Authentication: Failed`	A plaintext registration reached the public health RPC but supplied no identity to protected RPCs	Run `bash scripts/ensure_openshell_gateway.sh`. It uses the package service and re-registers `https://127.0.0.1:17670` with the generated local mTLS bundle. Do not accept `Connected` by itself.
Package gateway loops with `no compute driver configured` while `docker info` works in the shell	The long-running systemd user manager was created before the user received Docker-group membership, so its service children cannot open `/var/run/docker.sock`	Fully end all login sessions and log back in, or reboot, then rerun `bash scripts/ensure_openshell_gateway.sh`. If `loginctl show-user "$USER" -p Linger --value` is `yes`, use a maintenance reboot because logout does not recreate the lingering manager. Restarting only the gateway, `newgrp`, changing socket permissions, or running the gateway as root does not repair the manager's credentials.
Package gateway works over SSH but stops after disconnect	The host is using a systemd user service with linger disabled, so the last logout stops the user manager and all its services	After the gateway passes its checks, run `sudo loginctl enable-linger "$USER"` and verify `loginctl show-user "$USER" -p Linger --value` prints `yes`. This is the native OpenShell Linux service model.
OpenShell gateway service does not start	Package service failed or its HTTPS registration is stale	Check `systemctl --user status openshell-gateway` and `journalctl --user -u openshell-gateway --no-pager -n 50`, then rerun the setup helper.
`openshell sandbox create` fails with "port already forwarded" or hangs on `--forward 18789`	Stale port forward from a previously deleted sandbox is still registered	List forwards: `openshell forward list`. Stop each one bound to `:18789`: `openshell forward stop 18789 <sandbox-name>`. `setup_sandbox.sh` does this automatically before re-creating the sandbox.
Stale OpenShell gateway from another playbook is still registered	A previous playbook selected another gateway name or a legacy plaintext endpoint	Run the setup helper to select the managed `openshell` HTTPS/mTLS registration. Use `openshell gateway list -o json` to inspect registrations; do not kill the shared package service.
Port 18789 not accessible remotely	SSH tunnel not active or port forward dead inside sandbox	Check with `openshell forward list`. If dead: `openshell forward stop 18789 clinical-sandbox && openshell forward start -d 18789 clinical-sandbox`. Then re-establish SSH tunnel from your machine.
`requests` library doesn't work in sandbox	Sandbox Python uses curl subprocess for HTTP, not the requests library	This is by design. All HTTP calls in agent scripts must use `subprocess.run(["curl", ...])` and `json.loads()`. The `fhir_helpers.py` library handles this automatically.

Inference and model

Symptom	Cause	Fix
Agent returns empty response or timeout	Model unloaded from GPU memory after idle timeout	Send a warmup message first. Check `OLLAMA_KEEP_ALIVE` is set to `4h` in docker-compose.yml.
`curl: (7) Failed to connect` to inference.local	OpenShell inference provider not configured or Ollama not running	Verify Ollama: `curl -sf http://localhost:${OLLAMA_PORT:-11434}/`. Re-run `make setup` — it configures the inference provider automatically.
Sandbox cannot reach host Ollama (only Docker bridge IP times out)	Host Ollama's systemd unit binds to `127.0.0.1` by default	Add a systemd override binding to all interfaces: `sudo systemctl edit ollama` and insert `[Service]` then `Environment="OLLAMA_HOST=0.0.0.0"`, then `sudo systemctl daemon-reload && sudo systemctl restart ollama`. Docker Ollama (the default in this playbook) already binds to `0.0.0.0`.
OpenFold3 returns error for molecular visualization	Protein sequence too long or malformed input	OpenFold3 supports sequences up to 4096 amino acids (PyTorch backend) or 2048 (TensorRT). Check the protein sequence in `build_viewer.py`'s drug-target table.

Agent and skills

Symptom	Cause	Fix
`make setup` fails	Setup did not complete successfully	Re-run `make setup` — the script recreates the sandbox from scratch with fresh config. Ensure you're on OpenShell >= 0.0.111.
`make check` shows stale skills	Workspace skill copies don't match the repo after an update	The check output tells you which skills are stale. Re-run `make setup` or manually copy from `/sandbox/clinical-intelligence/skills/` to `~/.openclaw/workspace/skills/` inside the sandbox.
ENOENT errors for memory files in logs	OpenClaw tries to read daily memory files that don't exist	Create the memory directory: `mkdir -p ~/.openclaw/workspace/memory && touch ~/.openclaw/workspace/MEMORY.md` inside the sandbox. `make check` detects this.
Agent writes code from scratch instead of using helpers	Stale IDENTITY.md or analysis-methods skill in workspace	Run `make check` to verify. If stale, the workspace IDENTITY.md doesn't have the `fhir_helpers` import instruction.
Agent uses wrong LOINC code for eGFR	Agent used its own training knowledge instead of reading the skill file	Run `make check` to verify skills are synced. The fhir-basics skill lists `33914-3` for eGFR. If the workspace copy is stale, the model uses its own (often wrong) LOINC codes.

Demo and queries

Symptom	Cause	Fix
FHIR queries return 0 patients	Wrong SNOMED code format	Use bare codes: `code=44054006`, not `code=http://snomed.info/sct\|44054006`. The skill files contain the correct patterns.
Charts not visible in dashboard	Canvas directory not accessible or file not saved to correct path	Charts must be saved to `~/.openclaw/canvas/`. View canvas at `http://localhost:18789/__openclaw__/canvas/`.
`make test-full` fails on L4/L5 agent tests	Agent query timed out, FHIR server unreachable from sandbox, or Ollama model unloaded	Check step by step: (1) `make status` — are Ollama and OpenFold3 healthy? (2) `make check` — are skills and config synced? (3) Send a warmup message in the dashboard to reload the model. (4) Run `make test --level 3` first to isolate whether the issue is infrastructure, config, or agent-level.

OpenFold3: GPU not recognized

Older OpenFold3 NIM images selected a compute profile using the GPU's PCI device ID. A Blackwell SKU missing from that image's model_manifest.yaml could therefore fail during startup with NIMProfileIDNotFound.

The playbook now pins the OpenFold3 1.6.0 multi-architecture image by digest. Its bundled manifest uses a generic OpenFold3 profile rather than a PCI-ID allowlist. On a DGX Station GB300 (31c2:10de), the pinned image selected that profile, initialized CUDA worker 0, reached /v1/health/ready, and completed the bundled 8eil prediction. Pinning the digest also prevents an existing installation from silently resolving latest to a different image.

The public NIM/API release reported by /v1/version is 1.6.0. The image's internal manifest can report a later packaging revision; that does not indicate a different OpenFold model or an image-tag mismatch.

Recover an installation that still reports NIMProfileIDNotFound:

Bash

# Authenticate, pull the exact compose image, and replace the old container.
make ngc-login
docker compose pull openfold3
docker compose up -d --force-recreate openfold3

# OpenFold3 can take roughly three minutes to compile kernels and warm up.
docker compose logs -f openfold3
curl -sf http://localhost:${OPENFOLD_PORT:-8000}/v1/health/ready
curl -sf http://localhost:${OPENFOLD_PORT:-8000}/v1/version

If the pinned image still fails, preserve docker compose logs openfold3 and the output of docker image inspect for escalation. Do not rewrite or mount over the signed image manifest: mapping an unknown PCI ID to another profile is not a validated hardware fix and can select an incompatible profile.

For latest known issues, see the documentation linked under Resources for your hardware platform.