Lipsync

Generate API Key

Pull and Run the NIM

NVIDIA LipSync NIM uses gRPC APIs for inferencing requests.

A NGC API Key is required to download the appropriate models and resources when starting the NIM. Pass the value of the API Key to the docker run command in the next section as the NGC_API_KEY environment variable as indicated.

If you are not familiar with how to create the NGC_API_KEY environment variable, the simplest way is to export it in your terminal:

export NGC_API_KEY=<PASTE_API_KEY_HERE>

Run one of the following commands to make the key available at startup:

# If using bash
echo "export NGC_API_KEY=<value>" >> ~/.bashrc

# If using zsh
echo "export NGC_API_KEY=<value>" >> ~/.zshrc

Other, more secure options include saving the value in a file, so that you can retrieve with cat $NGC_API_KEY_FILE, or using a password manager.

To pull the NIM container image from NGC, first authenticate with the NVIDIA Container Registry with the following command:

echo "$NGC_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin

The following command launches a container with the gRPC service.

docker run -it --rm --name=lipsync-nim \
  --runtime=nvidia \
  --gpus all \
  --shm-size=8GB \
  -e NGC_API_KEY=$NGC_API_KEY \
  -e NIM_HTTP_API_PORT=8000 \
  -e NIM_GRPC_API_PORT=8001 \
  -p 8000:8000 \
  -p 8001:8001 \
  nvcr.io/nim/nvidia/lipsync:latest

Please note, the flag --gpus all is used to assign all available GPUs to the docker container. To assign specific GPU to the docker container (in case of multiple GPUs available in your machine) use --gpus '"device=0,1,2..."'

If the command runs successfully, you get a response similar to the following.

[INFO AI4M BASE LOGGER 2026-02-24 20:11:29.059 PID:217] Using threading mode for gRPC service
[INFO AI4M BASE LOGGER 2026-02-24 20:11:29.060 PID:217] Starting threading gRPC service with 1 threads
[INFO AI4M BASE LOGGER 2026-02-24 20:11:29.068 PID:217] Using Insecure Server Credentials
[INFO AI4M BASE LOGGER 2026-02-24 20:11:29.069 PID:217] Listening to 0.0.0.0:8001

By default LipSync gRPC service is hosted on port 8001. You will have to use this port for inferencing requests.

Test the NIM

We have provided a sample client script file in our GitHub repo. The script could be used to invoke the Docker container using the following instructions.

Download the LipSync Python client code by cloning the NIM Client Repository:

git clone https://github.com/NVIDIA-Maxine/nim-clients.git
cd nim-clients/lipsync

Install the dependencies for the lipSync gRPC client:

sudo apt-get install python3-pip
pip install -r requirements.txt

Go to scripts directory

cd scripts

Run the command to send gRPC request

python3 lipsync.py \
  --target <server_ip:port> \
  --video-input <input_video_file_path> \
  --audio-input <input_audio_file_path> \
  --output <output_file_path_and_name> \

Example command with sample input:

python3 lipsync.py --target 127.0.0.1:8001 --video-input ../assets/sample_video.mp4 --audio-input ../assets/sample_audio.wav --output out.mp4 

For more details on getting started with this NIM including configuring using parameters, visit the NVIDIA LipSync NIM Docs.