cuopt Model by NVIDIA

Follow the steps below to download and run the NVIDIA NIM inference microservice for this model on your infrastructure of choice.

Step 1
Generate API Key

Step 2
Pull and Run the NIM

Export NGC_API_KEY variable.

Bash

export NGC_API_KEY=<PASTE_API_KEY_HERE>

Run the NIM container with the following commands.

Bash

$ docker login nvcr.io
Username: $oauthtoken
Password: <PASTE_API_KEY_HERE>

For CUDA 12.9 and Python 3.13:

Bash

docker run -it \
    --gpus='"device=0"' \
    -p 5000:5000 \
    -e NGC_API_KEY \
    nvcr.io/nvidia/cuopt/cuopt:latest-cuda12.9-py3.13

For CUDA 13.0 and Python 3.13:

Bash

docker run -it \
    --gpus='"device=0"' \
    -p 5000:5000 \
    -e NGC_API_KEY \
    nvcr.io/nvidia/cuopt/cuopt:latest-cuda13.0-py3.13

This command will start the NIM container and expose port 5000 for the user to interact with the NIM.

Open a new terminal, leaving the terminal open with the just launched service. In the new terminal, wait until the health check end point returns {"status":"RUNNING","version":"<VERSION>"} before proceeding. This may take a couple of minutes. You can use the following command to query the health check.

Bash

curl http://localhost:5000/v2/health/ready

Step 3
Test the NIM

Python client example

Save following Python example to a file named nim_client.py.

Python

#!/usr/bin/env python3
import requests
import time

data = {"cost_matrix_data": {"data": {"0": [[0,1],[1,0]]}},
           "task_data": {"task_locations": [0,1]},
           "fleet_data": {"vehicle_locations": [[0,0],[0,0]]}}

response = requests.post(
    url="http://localhost:5000/cuopt/request",
    json=data
)
response_body = response.json()

poll_interval = 2  # Time in seconds between polls

solution_url = "http://localhost:5000/cuopt/solution/" + response_body["reqId"]

while True:
    response = requests.get(solution_url)
    if response.status_code == 200 and "response" in response.json().keys():
        print(response.json())
        break
    elif response.status_code == 200:
        print(f"Polling for response")
    else:
        response.raise_for_status()
    time.sleep(poll_interval)

Execute the example.

Bash

chmod +x nim_client.py

./nim_client.py

Shell client example

Save the following Shell example to a file named nim_client.sh.

Bash

#!/usr/bin/env bash
set -e

pip install jq

REQUEST_URL=http://localhost:5000/cuopt/request
SOLUTION_URL=http://localhost:5000/cuopt/solution

data='{"cost_matrix_data": {"data": {"0": [[0,1],[1,0]]}},
           "task_data": {"task_locations": [0,1]},
           "fleet_data": {"vehicle_locations": [[0,0],[0,0]]}}'
reqId=$(curl -H 'Content-Type: application/json' \
        -d "$data" "$REQUEST_URL" | jq -r '.reqId'
       )

echo $reqId

while true; do
  response=$(curl -s "$SOLUTION_URL/$reqId")

  if echo "$response" | jq -e '.response.solver_response.status == 0' > /dev/null; then
    echo "$response"
    break
  fi

  echo "Polling for response"

  sleep 2

  done

Execute the example.

Bash

chmod +x nim_client.sh

./nim_client.sh

The NIM displays the results to the terminal in JSON format.

For more details on getting started with this NIM, visit the NVIDIA CUOPT Docs

cuopt

Step 1Generate API Key

Step 2Pull and Run the NIM

Step 3Test the NIM

Python client example

Shell client example

Step 1
Generate API Key

Step 2
Pull and Run the NIM

Step 3
Test the NIM