nvidia/audio2face-3d

RUN ANYWHERE

Converts streamed audio to facial blendshapes for realtime lipsyncing and facial performances.

Audio-to-Face
Digital Humans
NVIDIA NIM
Speech-to-Animation
API Reference

Legal

By running the below commands, you accept the NVIDIA AI Enterprise Terms of Use and the NVIDIA Community Models License.

NVIDIA Audio2Face-3D NIM and Authoring Microservice allow you to upload audio files to drive an animation. NVIDIA will only use and store the audio files to provide you with the NVIDIA Audio2Face Authoring Microservice. For more information about our data processing practices, see our Privacy Policy. By clicking “Get API Key” you consent to the processing of your data in accordance with the NVIDIA Cloud Agreement and Service-Specific Terms for NVIDIA Audio2Face 3D Authoring Microservice and NVIDIA Audio2Face 3D Microservice NIM.

Pull and run nvidia/audio2face-3d using Docker (this will download the full model and run it in your local environment)

$ docker login nvcr.io
Username: $oauthtoken
Password: <PASTE_API_KEY_HERE>

Start NIM

Pull and run the NVIDIA NIM with the command below.

GPU specific optimized models are available for select GPUs. For using optimized models, refer the Supported Models and specify NIM_MANIFEST_PROFILE according to your GPU in the Docker run command below. You can also use -e NIM_DISABLE_MODEL_DOWNLOAD=true to generate models at deployment time.

export NGC_API_KEY=<PASTE_API_KEY_HERE>

export CONTAINER_NAME=audio2face
docker run -it --rm --name=$CONTAINER_NAME \
  --gpus all \
  --network=host \
  -e NGC_API_KEY=$NGC_API_KEY \
  -e NIM_MANIFEST_PROFILE=$NIM_MANIFEST_PROFILE \
  nvcr.io/nim/nvidia/audio2face-3d:1.2

This command will start the NIM container and expose port 52000 for the user to interact with the NIM.

It may take up to 30 minutes depending on your network speed, for the container to be ready and start accepting requests from the time the docker container is started.

When starting the service, you might encounter warnings labeled as GStreamer-WARNING. These warnings are safe to ignore as they are not used by Audio2Face-3D.

For more details on getting started with this NIM, visit the NVIDIA Audio2Face-3D Docs