---
title: "Generate Images and Videos with ComfyUI — Video Gen Workflow"
canonical: "https://build.nvidia.com/spark/comfyui/video-gen-workflow.md"
---

> [!NOTE]
> These instructions target **Linux**. This tab needs substantially more memory than **Image Gen Quick Start**:
>
> | Tier | Disk (weights) | Peak GPU memory (approx.) |
> |------|----------------|---------------------------|
> | **1 — Getting Started** | ~70 GB | ~80 GB (Wan 720p) |
> | **2 — Intermediate** | ~180 GB | ~100 GB |
> | **3 — Advanced** | ~230 GB | ~120 GB (Hunyuan 1080p) |
>
> Match the tier to your hardware platform’s available memory. On unified-memory platforms, start with Tier 1 and reduce resolution or frame count if you hit memory pressure. For a lighter single-image start (~24 GB class), use the **Image Gen Quick Start** tab.

# Step 1. Verify your environment

```bash
nvidia-smi
docker --version
df -h /
```

Expected: a detected NVIDIA GPU, Docker 24+, enough free disk for your model tier (Tier 1 ~70 GB; all tiers ~230 GB), and enough GPU memory for that tier’s peak (Tier 1 ~80 GB; see the note above).

If you have not already added your user to the docker group:

```bash
sudo usermod -aG docker $USER
newgrp docker
```

# Step 2. Set Hugging Face credentials

```bash
# Required for gated models. Run in the same shell as the download script.
# Get a token from https://huggingface.co/settings/tokens
export HF_TOKEN="your_huggingface_token"
```

Accept model licenses when prompted on Hugging Face, for example:

- [FLUX.1 dev](https://huggingface.co/black-forest-labs/FLUX.1-dev)
- [HiDream-I1 Full](https://huggingface.co/HiDream-ai/HiDream-I1-Full)

# Step 3. Clone this playbook and build the image

```bash
git clone https://github.com/NVIDIA/dgx-spark-playbooks
cd dgx-spark-playbooks/nvidia/playbook-comfyui
docker build -t comfyui -f assets/Dockerfile .
```

The build clones ComfyUI, installs dependencies (preserving the NGC-optimized PyTorch), and pre-installs custom nodes for video, ControlNet, and IP-Adapter. Expect about 5–10 minutes.

# Step 4. Download models by tier

| Tier | Models | Disk space | Peak VRAM (approx.) | Workflows enabled |
|------|--------|------------|---------------------|-------------------|
| **1 — Getting Started** | FLUX.1 dev, Wan 2.1 T2V 14B | ~70 GB | ~80 GB (Wan 720p clip) | Text-to-image, text-to-video |
| **2 — Intermediate** | + HiDream-I1, Wan 2.1 I2V, Cosmos-Predict2 | ~180 GB | ~100 GB (FLUX→Wan two-model graph) | + HiDream, image-to-video, FLUX→Wan, Cosmos Video2World |
| **3 — Advanced** | + HunyuanVideo, FLUX ControlNet (Canny) | ~230 GB | ~120 GB (Hunyuan 1080p / long clips) | + 1080p video, ControlNet-guided generation |

Peak VRAM depends on resolution, frame count, and precision. Match the tier to your hardware platform's available memory. On unified-memory platforms, large video models may need reduced resolution or frames.

Install the Hugging Face Hub CLI if needed:

```bash
pip3 install --break-system-packages huggingface-hub
echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc
export PATH="$HOME/.local/bin:$PATH"
hf --version
```

```bash
# Tier 1 only:
bash assets/scripts/download-models.sh 1

# All tiers:
bash assets/scripts/download-models.sh
```

After Tier 1 completes:

```bash
ls -la ./models/diffusion_models/
ls -la ./models/text_encoders/ | head
```

# Step 5. Launch the container

Identify the target GPU index when more than one GPU is present:

```bash
nvidia-smi --query-gpu=index,name --format=csv,noheader
```

Use `--gpus '"device=N"'` with that index (default `0` on single-GPU systems). `--gpus all` is fine when only one GPU is available; on multi-GPU systems, pin the device you intend to use.

```bash
docker run -d \
--name comfyui \
--gpus '"device=<GPU_ID>"' \
--ipc host \
--ulimit memlock=-1 \
-p 8188:8188 \
-v "$(pwd)/models:/opt/ComfyUI/models" \
-v "$(pwd)/output:/opt/ComfyUI/output" \
-v "$(pwd)/input:/opt/ComfyUI/input" \
-v "$(pwd)/assets/workflows:/opt/ComfyUI/user/default/workflows" \
-v "$HOME/.cache/huggingface:/root/.cache/huggingface" \
comfyui
```

Follow logs until the UI is ready:

```bash
docker logs -f comfyui
```

Expected ready line:

```
To see the GUI go to: http://0.0.0.0:8188
```

Press `Ctrl+C` to leave the log view.

> [!NOTE]
> Startup may print benign warnings (CUDA-hooks diagnostics, package version skew, missing optional audio/ONNX GPU wheels). Treat the `To see the GUI go to: ...` line as the ready signal.

# Step 6. UI workflows vs API graphs

| Location | Format | Use |
|----------|--------|-----|
| `assets/workflows/*.json` (mounted into the UI workflow folder) | UI workflow (`nodes` / `links`) | **Load** in the web UI, then **Queue Prompt** |
| `assets/workflow_api/*.api.json` | API prompt graph | `POST /prompt`, `curl`, automation |

Loading an `.api.json` file with **Load** shows **"Error: the workflow does not contain any nodes"** — expected; those files are for the HTTP API only.

Optional HTTP API example (from the playbook root, ComfyUI on port 8188):

```bash
PROMPT=$(python3 -c "import json; d=json.load(open('assets/workflow_api/flux-text-to-image.api.json')); print(json.dumps({k:v for k,v in d.items() if str(k).isdigit()}, separators=(',',':')))")
curl -sS http://127.0.0.1:8188/prompt \
-X POST \
-H "Content-Type: application/json" \
-d "{\"prompt\":${PROMPT}}" | python3 -m json.tool
```

# Step 7. Validate the install

```bash
curl -I http://localhost:8188
```

Expected: HTTP 200. Open `http://<HARDWARE_IP>:8188` in a browser (`<HARDWARE_IP>` is the IP of your hardware platform).

**FLUX text-to-image (Tier 1):** **Load** `flux-text-to-image.json`, enter a prompt in **CLIP Text Encode**, click **Queue Prompt**. Expect roughly 15–30 seconds at default settings on high-memory hardware.

# Step 8. Image and video workflows

*Requires the matching model tier from Step 4.*

## Text-to-video with Wan 2.1 (Tier 1)

Load `wan-text-to-video.json`. Default graph targets ~720p, 81 frames (~5 s). Generation can take several minutes. Reduce frame count for faster iteration. Convert animated WEBP output to MP4 with `ffmpeg` if needed.

## Intermediate workflows (Tier 2)

- `hidream-text-to-image.json` — HiDream-I1 Full (17B) with four text encoders
- `wan-image-to-video.json` — place a source image in `input/` first
- `flux-to-wan-pipeline.json` — FLUX still → Wan I2V in one graph
- `cosmos-video2world.json` — NVIDIA Cosmos-Predict2 Video2World from an input image

## Advanced workflows (Tier 3)

- `hunyuan-1080p-video.json` — 1080p-class video (height divisible by 16; default 1920×1056). Needs very large GPU memory (~100–120 GB at default settings).
- `flux-controlnet.json` — Canny-conditioned FLUX; place a reference image in `input/`

# Step 9. Optional — cleanup and rollback

> [!WARNING]
> Cleanup removes the container and optionally downloaded models. Generated outputs in `output/` are preserved unless you delete them.

```bash
docker stop comfyui
docker rm comfyui
docker rmi comfyui          # optional
# sudo may be required if the container wrote files as root:
# sudo rm -rf models/
```

To avoid root-owned files on future runs, add `--user "$(id -u):$(id -g)"` to `docker run` (host UID must be able to write mounted directories).