Generate Images and Videos with ComfyUI

NOTE

These instructions target Linux. This tab needs substantially more memory than Image Gen Quick Start:

Tier	Disk (weights)	Peak GPU memory (approx.)
1 — Getting Started	~70 GB	~80 GB (Wan 720p)
2 — Intermediate	~180 GB	~100 GB
3 — Advanced	~230 GB	~120 GB (Hunyuan 1080p)

Match the tier to your hardware platform’s available memory. On unified-memory platforms, start with Tier 1 and reduce resolution or frame count if you hit memory pressure. For a lighter Stable Diffusion–only start (~8 GB class), use the Image Gen Quick Start tab.

Step 1
Verify your environment

Bash

nvidia-smi
docker --version
df -h /

Expected: a detected NVIDIA GPU, Docker 24+, enough free disk for your model tier (Tier 1 ~70 GB; all tiers ~230 GB), and enough GPU memory for that tier’s peak (Tier 1 ~80 GB; see the note above).

If you have not already added your user to the docker group:

Bash

sudo usermod -aG docker $USER
newgrp docker

Step 2
Set Hugging Face credentials

Bash

# Required for gated models. Run in the same shell as the download script.
# Get a token from https://huggingface.co/settings/tokens
export HF_TOKEN="your_huggingface_token"

Accept model licenses when prompted on Hugging Face, for example:

Step 3
Clone this playbook and build the image

Bash

git clone https://github.com/NVIDIA/dgx-spark-playbooks
cd dgx-spark-playbooks/nvidia/playbook-comfyui
docker build -t comfyui -f assets/Dockerfile .

The build clones ComfyUI, installs dependencies (preserving the NGC-optimized PyTorch), and pre-installs custom nodes for video, ControlNet, and IP-Adapter. Expect about 5–10 minutes.

Step 4
Download models by tier

Tier	Models	Disk space	Peak VRAM (approx.)	Workflows enabled
1 — Getting Started	FLUX.1 dev, Wan 2.1 T2V 14B	~70 GB	~80 GB (Wan 720p clip)	Text-to-image, text-to-video
2 — Intermediate	+ HiDream-I1, Wan 2.1 I2V, Cosmos-Predict2	~180 GB	~100 GB (FLUX→Wan two-model graph)	+ HiDream, image-to-video, FLUX→Wan, Cosmos Video2World
3 — Advanced	+ HunyuanVideo, FLUX ControlNet (Canny)	~230 GB	~120 GB (Hunyuan 1080p / long clips)	+ 1080p video, ControlNet-guided generation

Peak VRAM depends on resolution, frame count, and precision. Match the tier to your hardware platform's available memory. On unified-memory platforms, large video models may need reduced resolution or frames.

Install the Hugging Face Hub CLI if needed:

Bash

pip3 install --break-system-packages huggingface-hub
echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc
export PATH="$HOME/.local/bin:$PATH"
hf --version

Bash

# Tier 1 only:
bash assets/scripts/download-models.sh 1

# All tiers:
bash assets/scripts/download-models.sh

After Tier 1 completes:

Bash

ls -la ./models/diffusion_models/
ls -la ./models/text_encoders/ | head

Step 5
Launch the container

Identify the target GPU index when more than one GPU is present:

Bash

nvidia-smi --query-gpu=index,name --format=csv,noheader

Use --gpus '"device=N"' with that index (default 0 on single-GPU systems). --gpus all is fine when only one GPU is available; on multi-GPU systems, pin the device you intend to use.

Bash

docker run -d \
  --name comfyui \
  --gpus '"device=<GPU_ID>"' \
  --ipc host \
  --ulimit memlock=-1 \
  -p 8188:8188 \
  -v "$(pwd)/models:/opt/ComfyUI/models" \
  -v "$(pwd)/output:/opt/ComfyUI/output" \
  -v "$(pwd)/input:/opt/ComfyUI/input" \
  -v "$(pwd)/assets/workflows:/opt/ComfyUI/user/default/workflows" \
  -v "$HOME/.cache/huggingface:/root/.cache/huggingface" \
  comfyui

Follow logs until the UI is ready:

Bash

docker logs -f comfyui

Expected ready line:

Bash

To see the GUI go to: http://0.0.0.0:8188

Press Ctrl+C to leave the log view.

NOTE

Startup may print benign warnings (CUDA-hooks diagnostics, package version skew, missing optional audio/ONNX GPU wheels). Treat the To see the GUI go to: ... line as the ready signal.

Step 6
UI workflows vs API graphs

Location	Format	Use
`assets/workflows/*.json` (mounted into the UI workflow folder)	UI workflow (`nodes` / `links`)	Load in the web UI, then Queue Prompt
`assets/workflow_api/*.api.json`	API prompt graph	`POST /prompt`, `curl`, automation

Loading an .api.json file with Load shows "Error: the workflow does not contain any nodes" — expected; those files are for the HTTP API only.

Optional HTTP API example (from the playbook root, ComfyUI on port 8188):

Bash

PROMPT=$(python3 -c "import json; d=json.load(open('assets/workflow_api/flux-text-to-image.api.json')); print(json.dumps({k:v for k,v in d.items() if str(k).isdigit()}, separators=(',',':')))")
curl -sS http://127.0.0.1:8188/prompt \
  -X POST \
  -H "Content-Type: application/json" \
  -d "{\"prompt\":${PROMPT}}" | python3 -m json.tool

Step 7
Validate the install

Bash

curl -I http://localhost:8188

Expected: HTTP 200. Open http://<HARDWARE_IP>:8188 in a browser (<HARDWARE_IP> is the IP of your hardware platform).

FLUX text-to-image (Tier 1): Load flux-text-to-image.json, enter a prompt in CLIP Text Encode, click Queue Prompt. Expect roughly 15–30 seconds at default settings on high-memory hardware.

Step 8
Image and video workflows

Requires the matching model tier from Step 4.

Text-to-video with Wan 2.1 (Tier 1)

Load wan-text-to-video.json. Default graph targets ~720p, 81 frames (~5 s). Generation can take several minutes. Reduce frame count for faster iteration. Convert animated WEBP output to MP4 with ffmpeg if needed.

Intermediate workflows (Tier 2)

hidream-text-to-image.json — HiDream-I1 Full (17B) with four text encoders
wan-image-to-video.json — place a source image in input/ first
flux-to-wan-pipeline.json — FLUX still → Wan I2V in one graph
cosmos-video2world.json — NVIDIA Cosmos-Predict2 Video2World from an input image

Advanced workflows (Tier 3)

hunyuan-1080p-video.json — 1080p-class video (height divisible by 16; default 1920×1056). Needs very large GPU memory (~100–120 GB at default settings).
flux-controlnet.json — Canny-conditioned FLUX; place a reference image in input/

Step 9
Optional — cleanup and rollback

WARNING

Cleanup removes the container and optionally downloaded models. Generated outputs in output/ are preserved unless you delete them.

Bash

docker stop comfyui
docker rm comfyui
docker rmi comfyui          # optional
# sudo may be required if the container wrote files as root:
# sudo rm -rf models/

To avoid root-owned files on future runs, add --user "$(id -u):$(id -g)" to docker run (host UID must be able to write mounted directories).

Generate Images and Videos with ComfyUI

Step 1Verify your environment

Step 2Set Hugging Face credentials

Step 3Clone this playbook and build the image

Step 4Download models by tier

Step 5Launch the container

Step 6UI workflows vs API graphs

Step 7Validate the install

Step 8Image and video workflows

Text-to-video with Wan 2.1 (Tier 1)

Intermediate workflows (Tier 2)

Advanced workflows (Tier 3)

Step 9Optional — cleanup and rollback

Resources

Step 1
Verify your environment

Step 2
Set Hugging Face credentials

Step 3
Clone this playbook and build the image

Step 4
Download models by tier

Step 5
Launch the container

Step 6
UI workflows vs API graphs

Step 7
Validate the install

Step 8
Image and video workflows

Step 9
Optional — cleanup and rollback