Generate Images and Videos with ComfyUI

Install with an agent

Use https://build.nvidia.com/playbooks/comfyui.md and complete this playbook on this machine: Generate Images and Videos with ComfyUI. Fetch that .md URL directly; the non-.md page is a JavaScript shell. Cover only the Video Gen Workflow (Tier 1 models). When finished, state clearly whether the playbook succeeded, and give the evidence: what you installed, where, and the output of a command proving it works (a version string, a container listing, or an HTTP status).

Verify your environment

nvidia-smi
docker --version
df -h /

Expected: a detected NVIDIA GPU, Docker 24+, enough free disk for your model tier (Tier 1 ~70 GB; all tiers ~230 GB), and enough GPU memory for that tier’s peak (Tier 1 ~80 GB; see the note above).

If you have not already added your user to the docker group:

sudo usermod -aG docker $USER
newgrp docker

Set Hugging Face credentials

# Required for gated models. Run in the same shell as the download script.
# Get a token from https://huggingface.co/settings/tokens
export HF_TOKEN="your_huggingface_token"

Accept model licenses when prompted on Hugging Face, for example:

• FLUX.1 dev
• HiDream-I1 Full

Clone this playbook and build the image

git clone https://github.com/NVIDIA/dgx-spark-playbooks
cd dgx-spark-playbooks/nvidia/playbook-comfyui
docker build -t comfyui -f assets/Dockerfile .

The build clones ComfyUI, installs dependencies (preserving the NGC-optimized PyTorch), and pre-installs custom nodes for video, ControlNet, and IP-Adapter. Expect about 5–10 minutes.

Download models by tier

Peak VRAM depends on resolution, frame count, and precision. Match the tier to your hardware platform's available memory. On unified-memory platforms, large video models may need reduced resolution or frames.

Install the Hugging Face Hub CLI if needed:

pip3 install --break-system-packages huggingface-hub
echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc
export PATH="$HOME/.local/bin:$PATH"
hf --version

# Tier 1 only:
bash assets/scripts/download-models.sh 1

# All tiers:
bash assets/scripts/download-models.sh

After Tier 1 completes:

ls -la ./models/diffusion_models/
ls -la ./models/text_encoders/ | head

Launch the container

Identify the target GPU index when more than one GPU is present:

nvidia-smi --query-gpu=index,name --format=csv,noheader

Use --gpus '"device=N"' with that index (default 0 on single-GPU systems). --gpus all is fine when only one GPU is available; on multi-GPU systems, pin the device you intend to use.

docker run -d \
  --name comfyui \
  --gpus '"device=<GPU_ID>"' \
  --ipc host \
  --ulimit memlock=-1 \
  -p 8188:8188 \
  -v "$(pwd)/models:/opt/ComfyUI/models" \
  -v "$(pwd)/output:/opt/ComfyUI/output" \
  -v "$(pwd)/input:/opt/ComfyUI/input" \
  -v "$(pwd)/assets/workflows:/opt/ComfyUI/user/default/workflows" \
  -v "$HOME/.cache/huggingface:/root/.cache/huggingface" \
  comfyui

  comfyui python main.py --listen 0.0.0.0 --port 8188 --disable-dynamic-vram

Follow logs until the UI is ready:

docker logs -f comfyui

Expected ready line:

To see the GUI go to: http://0.0.0.0:8188

Press Ctrl+C to leave the log view.

UI workflows vs API graphs

Loading an .api.json file with Load shows "Error: the workflow does not contain any nodes" — expected; those files are for the HTTP API only.

Optional HTTP API example (from the playbook root, ComfyUI on port 8188):

PROMPT=$(python3 -c "import json; d=json.load(open('assets/workflow_api/flux-text-to-image.api.json')); print(json.dumps({k:v for k,v in d.items() if str(k).isdigit()}, separators=(',',':')))")
curl -sS http://127.0.0.1:8188/prompt \
  -X POST \
  -H "Content-Type: application/json" \
  -d "{\"prompt\":${PROMPT}}" | python3 -m json.tool

Validate the install

curl -I http://localhost:8188

Expected: HTTP 200. Open http://<HARDWARE_IP>:8188 in a browser (<HARDWARE_IP> is the IP of your hardware platform).

FLUX text-to-image (Tier 1): Load flux-text-to-image.json, enter a prompt in CLIP Text Encode, click Queue Prompt. Expect roughly 15–30 seconds at default settings on high-memory hardware.

Image and video workflows

Requires the matching model tier from Step 4.

Text-to-video with Wan 2.1 (Tier 1)

Load wan-text-to-video.json. Default graph targets ~720p, 81 frames (~5 s). Generation can take several minutes. Reduce frame count for faster iteration. Convert animated WEBP output to MP4 with ffmpeg if needed.

Intermediate workflows (Tier 2)

• hidream-text-to-image.json — HiDream-I1 Full (17B) with four text encoders
• wan-image-to-video.json — place a source image in input/ first
• flux-to-wan-pipeline.json — FLUX still → Wan I2V in one graph
• cosmos-video2world.json — NVIDIA Cosmos-Predict2 Video2World from an input image

Advanced workflows (Tier 3)

• hunyuan-1080p-video.json — 1080p-class video (height divisible by 16; default 1920×1056). Needs very large GPU memory (~100–120 GB at default settings).
• flux-controlnet.json — Canny-conditioned FLUX; place a reference image in input/

Optional — cleanup and rollback

docker stop comfyui
docker rm comfyui
docker rmi comfyui          # optional
# sudo may be required if the container wrote files as root:
# sudo rm -rf models/

To avoid root-owned files on future runs, add --user "$(id -u):$(id -g)" to docker run (host UID must be able to write mounted directories).