AI models generate responses and outputs based on complex algorithms and machine learning techniques, and those responses or outputs may be inaccurate, harmful, biased or indecent. By testing this model, you assume the risk of any harm caused by any response or output of the model. Please do not upload any confidential information or personal data unless expressly permitted. Your use is logged for security purposes.

Overview

The NVIDIA Retail Shopping Assistant is a reference conversational shopping advisor built on LangChain Deep Agents and NVIDIA Nemotron open models for reasoning, perception, and safety, with an optional NemoGuard topic control guardrail. Shoppers can type a request or upload a photo or short video. The agent plans each search, and a catalog service retrieves products deterministically through embedding search and structured filtering, with no generative model in the retrieval path. Cart changes go through typed tools, with totals computed in code. Each turn is a durable record, so its outcome and the products it showed survive a restart. The blueprint includes a representative catalog and five notebooks that run on a single deployment spanning setup, observability, evaluation, and GPU performance measurement. It runs on NVIDIA-hosted endpoints with no local GPU or on your own GPUs with the models served by vLLM.

Architecture Diagram

Key Features

• Text, Photo, and Video Shopping: Shoppers describe what they want or upload an image or short clip, and Nemotron 3.5 Super VL understands it in a shopping context.
• Agentic Search You Can Trust: A Deep Agents runtime plans each search, while a deterministic catalog service retrieves and ranks products. Product facts come from the catalog, and a grounding step keeps each reply to them.
• Controlled Tool Use: Each skill declares which tools it may call, and the grant is checked again at every call.
• Accurate Cart: Typed cart tools compute totals in code, with prices from the catalog, so totals and quantities are never generated.
• Conversations That Persist: Turns and the products shown are stored durably and survive a restart.
• Your Catalog, Your Filters: The catalog declares which fields become shopper-facing filters, so you can point the service at your own data.
• Configurable Guardrails: Nemotron 3.5 Content Safety, Llama 3.1 NemoGuard 8B Topic Control, and Nemotron 3 Nano Omni video safety check each turn. Guardrails are on by default for API requests, can be toggled per session in the UI, or left out entirely.
• Built-In Observability: Per-turn traces in Phoenix show which skill ran, what the model was told, and why a tool was refused.
• Flexible Deployment: Use NVIDIA-hosted endpoints with no GPU, or serve the Hugging Face checkpoints yourself with vLLM.
• Five Notebooks: Deployment, observability, evaluation, trace capture, and GPU performance measurement.

Minimum System Requirements

Hardware Requirements

• The blueprint uses NVIDIA-hosted API endpoints by default, so you can try it without local GPUs. A machine with Docker, 32 GB of RAM, and 50 GB of disk is enough.
• As your development progresses, you can self-host the models. The recommended system for running every model locally is six H100 80 GB GPUs, with 128 GB+ of system memory and about 300 GB of disk for the model checkpoints:
  • Full layout
    • GPUs 0–3: NVIDIA Nemotron 3.5 Super VL (BF16, tensor parallel 4), serving both the shopping agent and media understanding.
    • GPU 4, shared:
      • Nemotron 3.5 Embed 1B text embedding model
      • Nemotron 3.5 Content Safety guardrail
      • Llama 3.1 NemoGuard 8B Topic Control guardrail
    • GPU 5: Nemotron 3 Nano Omni 30B (FP8), the video safety guardrail.
  • Smaller layout
    • 5 GPUs: guardrails hosted, or no guardrails.

OS Requirements

• Ubuntu 22.04 OS

Deployment Options

• Docker

Software Used in This Blueprint

NVIDIA Technology

• Nemotron 3.5 Super VL (shopping agent, photo, and video)
• Nemotron 3.5 Embed 1B (embeddings)
• Nemotron 3.5 Content Safety and Llama 3.1 NemoGuard 8B Topic Control (optional guardrails)
• Nemotron 3 Nano Omni 30B A3B Reasoning (optional video safety guardrailmultimodal safety role)
• NeMo Relay (optional agent tracing)

3rd Party Software

• Deep Agents, LangChain, LangGraph, vLLM, FastAPI, React, Milvus, etcd, SeaweedFS, nginx, OpenTelemetry Collector, Arize Phoenix, and the Visual Crossing weather API (optional).

Ethical Considerations

NVIDIA believes Trustworthy AI is a shared responsibility, and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their supporting model team to ensure the models meet requirements for the relevant industry and use case and address unforeseen product misuse. For more detailed information on ethical considerations for the models, please see the Model Card++ Explainability, Bias, Safety & Security, and Privacy Subcards. Please report security vulnerabilities or NVIDIA AI concerns here.

License

GOVERNING DOWNLOAD TERMS: Use of this blueprint software is governed by the Apache 2.0 License, and enables use of separate open source and proprietary software, models, data and services governed by their respective accompanying licenses.

Terms of Use

This project will download and install additional third-party open source software projects, including vLLM, Milvus, LangChain, LangGraph, Deep Agents, NeMo Guardrails, FastAPI and React. Review the license terms of these open source projects before use, found in LICENSE-3rd-party.txt.