# USD Search ## Model Overview USD Search is an AI-powered search for OpenUSD data, three dimensional (3D) models, images, and assets using text or image-based inputs. It leverages NVCLIP, which is a NVIDIA commercial version of the "Contrastive Language-Image Pre-Training (CLIP)" model that transforms an image into textual embeddings. ## References: - Radford, Alec, et al. "Learning transferable visual models from natural language supervision." International conference on machine learning. PMLR, 2021. ## Model Architecture: **Architecture Type:** Transformer-based architecture
## Input: **Input Type(s):** Text or Image
**Input Format(s):** Text, or Red, Green, Blue (RGB)
**Other Properties Related to Input:** The model accepts either text or image input, but not both simultaneously
## Output: **Output Type(s):** List
**Output Format:** Rendered thumbnails, asset metadata
**Other Properties Related to Output:**
The output of this model is a sorted-by-relevance list of OpenUSD assets. List contains rendered thumbnails and associated metadata containing URL pointing to the location of the asset in the backend database. ## Software Integration: **Runtime Engine(s):** * TensorRT
**Supported Hardware Architecture(s):**
* NVIDIA Ampere
* NVIDIA Hopper
* NVIDIA Lovelace
**Supported Operating System(s):**
* Linux
## Model Version(s): - nv_clip_224_vit_h - NVCLIP ViT-H with 224 resolution. ## Training & Evaluation: No additional training or evaluation in addition to what has been done for the NVCLIP model. ## Using this Model These models need to be used with NVIDIA hardware and software. For hardware, the models can run on any of the latest NVIDIA GPUs since NVIDIA Ampere. ## Training Dataset: **Data Collection Method by dataset:**
* Automated
**Labeling Method by dataset:**
* Automated
**Properties:**
| Dataset | No. of Images | |--|--| |NV Internal Data| 700M | ## Evaluation Dataset: **Link:** [https://www.image-net.org/](https://www.image-net.org/) **Data Collection Method by dataset:**
* Unknown
**Labeling Method by dataset:**
* Unknown
**Properties:**
50,000 validation images from [ImageNet dataset](https://www.image-net.org/download.php)
The performance details of the underlying NVCLIP model is noted below. #### Methodology and KPI The performance of zero shot accuracy of NVCLIP on ImageNet validation dataset. | model | top-1 Accuracy | | ----------------------- | -------------- | | ViT-H-336 | 0.7786 | | ViT-L-336 | 0.7629 | ## Inference: **Engine:** TensorRT
**Test Hardware:**
- A100 - L40 ## Ethical Considerations: NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse. For more detailed information on ethical considerations for this model, please see the Model Card++ Promise and the Explainability, Bias, Safety & Security, and Privacy Subcards.

nvidia

usdsearch