---
title: "usdsearch"
publisher: "nvidia"
type: "endpoint"
updated: "2024-11-14T16:34:00.377Z"
description: "AI-powered search for OpenUSD data, 3D models, images, and assets using text or image-based inputs."
canonical: "https://build.nvidia.com/nvidia/usdsearch"
---

# USD Search

## Model Overview
USD Search is an AI-powered search for OpenUSD data, three dimensional (3D) models, images, and assets using text or image-based inputs. It leverages NVCLIP, which is a NVIDIA commercial version of the "Contrastive Language-Image Pre-Training (CLIP)" model that transforms an image into textual embeddings.

## References:

- Radford, Alec, et al. "Learning transferable visual models from natural language supervision." International conference on machine learning. PMLR, 2021.

## Model Architecture: 
**Architecture Type:** Transformer-based architecture  <br>

## Input:
**Input Type(s):** Text or Image <br> 
**Input Format(s):** Text, or Red, Green, Blue (RGB) <br>
**Other Properties Related to Input:** The model accepts either text or image input, but not both simultaneously <br>

## Output:
**Output Type(s):** List <br>
**Output Format:** Rendered thumbnails, asset metadata <br>
**Other Properties Related to Output:** <br> 
The output of this model is a sorted-by-relevance list of OpenUSD assets. List contains rendered thumbnails and associated metadata containing URL pointing to the location of the asset in the backend database. 

## Software Integration:
**Runtime Engine(s):** 
* TensorRT <br>

**Supported Hardware Architecture(s):** <br>
* NVIDIA Ampere <br>
* NVIDIA Hopper <br>
* NVIDIA Lovelace <br>

**Supported Operating System(s):** <br>
* Linux <br>

## Model Version(s): 
- nv_clip_224_vit_h - NVCLIP ViT-H with 224 resolution.

## Training & Evaluation: 

No additional training or evaluation in addition to what has been done for the NVCLIP model.

## Using this Model <a class="anchor" name="how_to_use_this_model"></a>

These models need to be used with NVIDIA hardware and software. For hardware, the models can run on any of the latest NVIDIA GPUs since NVIDIA Ampere. 

## Training Dataset:

**Data Collection Method by dataset:** <br>
* Automated <br>

**Labeling Method by dataset:** <br>
* Automated <br>

**Properties:**  <br>

| Dataset | No. of Images |
|--|--|
|NV Internal Data| 700M | 

## Evaluation Dataset:

**Link:** [https://www.image-net.org/](https://www.image-net.org/)

**Data Collection Method by dataset:** <br>
* Unknown <br>

**Labeling Method by dataset:** <br>
* Unknown <br>

**Properties:**  <br>
50,000 validation images from [ImageNet dataset](https://www.image-net.org/download.php) <br>
The performance details of the underlying NVCLIP model is noted below.

#### Methodology and KPI

The performance of zero shot accuracy of NVCLIP on ImageNet validation dataset.

| model                   | top-1 Accuracy |
| ----------------------- | -------------- |
| ViT-H-336               | 0.7786         |
| ViT-L-336               | 0.7629         |

## Inference:
**Engine:** TensorRT <br>
**Test Hardware:** <br>
- A100
- L40

## Ethical Considerations:

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications.  When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.  For more detailed information on ethical considerations for this model, please see the Model Card++ Promise and the Explainability, Bias, Safety & Security, and Privacy Subcards.

## Bias

| Field | Response |
| -- | -- |
|Participation considerations from adversely impacted groups [(protected classes)](https://www.senate.ca.gov/content/protected-classes) in model design and testing: | None of the Above |
| Measures taken to mitigate against unwanted bias: | Scaled Dataset and adding augmentations.|

## Explainability

| Field | Response |
| -- | -- |
| Intended Application(s) & Domain(s): | Generating image embedding that is aligned with text for zero-shot classification. |
| Model Type: | Embedding Generation |
| Intended Users: | This model is intended for developers building search engines, classification, detection/ segmentation models. |
| Output: | Embedding Features |
| Describe how the model works: | This model has a vision extractor and a text encoder trained for embedding alignment |
| Technical Limitations: | Model needs a downstream task specific head to perform CV tasks. |
| Verified to have met prescribed NVIDIA standards: | Yes |
| Performance Metrics: | ImageNet zero-shot accuracy  |
| Licensing: | GOVERNING TERMS: This trial is governed by the [NVIDIA API Trial Terms of Service](https://assets.ngc.nvidia.com/products/api-catalog/legal/NVIDIA%20API%20Trial%20Terms%20of%20Service.pdf). The use of this model is governed by the [AI Foundation Models Community License Agreement](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-ai-foundation-models-community-license-agreement/). |

## Privacy

| Field | Response |
| -- | -- |
| Generatable or reverse engineerable personally-identifiable information (PII)? | None |
| Protected classes used to create this model? | Not Applicable (No PII) |
| Was consent obtained for any personal data used? | Not Applicable (No personal data) |
| How often is dataset reviewed? | 	Before Release |
| Is a mechanism in place to honor data subject right of access or deletion of personal data? | No |
| If personal data collected for the development of the model, was it collected directly by NVIDIA? |Not Applicable |
| If personal data collected for the development of the model by NVIDIA, do you maintain or have access to disclosures made to data subjects?	| Not Applicable |
| If personal data collected for the development of this AI model, was it minimized to only what was required? | Not Applicable |
| Is there provenance for all datasets used in training? | Yes |
| Does data labeling (annotation, metadata) comply with privacy laws? | Yes |
| Is data compliant with data subject requests for data correction or removal, if such a request was made? | Yes |
| Applicable NVIDIA Privacy Policy	| [https://www.nvidia.com/en-us/data-center/products/nvidia-ai-enterprise/eula/](https://www.nvidia.com/en-us/data-center/products/nvidia-ai-enterprise/eula/) |

## Safety & Security

| Field | Response |
| -- | -- |
| Model Application(s): | Embedding generation and retrieval for OpenUSD assets |
| Describe the life-critical application (if present). | None: Not within Operational Design Domain |
| Use Case Restrictions: | Abide by [https://www.nvidia.com/en-us/data-center/products/nvidia-ai-enterprise/eula/](https://www.nvidia.com/en-us/data-center/products/nvidia-ai-enterprise/eula/)  |
| Describe access restrictions (if any): | The Principle of least privilege (PoLP) is applied limiting access for dataset generation and model development.  |