nvidia

3D Guided Generative AI

Create high quality images using Flux.1 in ComfyUI, guided by 3D.

Description

The 3D Guided Generative AI Blueprint unlocks greater control over image generation by laying out the content in Blender to guide the image layout. Users can quickly swap assets in the 3D scene using generative AI, and the perspective of the output image is switched by simply changing the camera angle.

The blueprint produces high-quality outputs by leveraging Black Forest Labs’ state-of-the-art FLUX.1-dev suite of models, and ComfyUI provides a flexible and convenient UI. The models are quantized and accelerated on NVIDIA GPUs, ensuring optimal performance.

The blueprint and corresponding instructions can be found on our AI Blueprints Github.

This blueprint is ready for non-commercial use. Contact sales@blackforestlabs.ai for commercial terms.

Terms of use

GOVERNING TERMS: The NIM container is governed by the NVIDIA Software License Agreement and Product-Specific Terms for AI Products. Contact sales@blackforestlabs.ai for commercial terms to use the Flux.1-dev model. ADDITIONAL INFORMATION: Apache 2.0, NVIDIA Community Model License Agreement and Llama 2 Community Model License Agreement.

Third-Party Community Consideration:

The models in this blueprint are not owned or developed by NVIDIA. They have been developed and built to a third-party’s requirements; see links to:

Architecture Diagram

Deployment Geography:

Global

Use Case:

Creators looking to improve and accelerate their workflows by leveraging generative AI
Developers exploring adding AI image generation capabilities to their applications

Software Integration:

Software Components: The blueprint does not directly contain any software, but it provides a script to download the following components:

ComfyUI
ComfyUI-BlenderAI node
Key software dependencies, such as Python and Git

Runtime Engines:

TensorRT

Supported GPUs:

GeForce RTX 5090
GeForce RTX 5080
GeForce RTX 4090
GeForce RTX 4080
GeForce RTX 4090 Laptop
NVIDIA RTX 6000 Lovelace Generation

Supported Operating Systems:

Windows Subsystem for Linux

Minimum System Requirements (for Windows)

VRAM: 16 GB
RAM: 48 GB

Model Version(s):

FLUX.1-dev
FLUX.1-Canny-dev
FLUX.1-Depth-dev
LiheYoung/Depth-anything-large-hf

Ethical Considerations:

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

Please report security vulnerabilities or NVIDIA AI Concerns here.