The NVIDIA AI Blueprint for RAG provides developers with a foundational starting point for building scalable and customizable retrieval pipelines that deliver high-accuracy and throughput. Use the blueprint to create a RAG application that delivers context-aware responses, connecting LLMs to large corpora of enterprise data, to enable actionable insights grounded in relevant data. The blueprint can be used as is, or combined with other NVIDIA Blueprints to address more advanced use cases including digital humans and AI assistants for customer service. Get started with this reference architecture to enhance decision-making and productivity.
Architecture Diagram
Key Features
- OpenAI-compatible APIs
- Multi-turn conversations
- Multi-collection
- Multi-session support
- Multilingual and cross-lingual retrieval
- Optimized data storage
- Configurability options for NIM selection and NIM endpoints
- Reranking usage
Minimum System Requirements
Hardware Requirements
- The blueprint by default uses API endpoints, making it very easy to experience without needing GPUs.
- It is expected that the NIM microservices will need to be self-hosted as you progress in your RAG development. For self-hosting the blueprint with these microservices locally deployed, the recommended system requirement is 5 H100 or A100 GPUs with the Llama 3.1 70b NIM, the NeMo Retriever embedding and reranking NIM, and the Milvus database accelerated with NVIDIA cuVS.
OS Requirements
- Ubuntu 22.04 OS
Deployment Options
- Docker
- Kubernetes
Software used in this blueprint
NIM microservices
NVIDIA Technology
- NeMo Retriever Llama 3.2 Embedding NIM
- NeMo Retriever Llama 3.2 Reranking NIM
- Llama 3.1 70B Instruct NIM
3rd Party Software
Ethical Considerations
NVIDIA believes Trustworthy AI is a shared responsibility, and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their supporting model team to ensure the models meet requirements for the relevant industry and use case and address unforeseen product misuse. For more detailed information on ethical considerations for the models, please see the Model Card++ Explainability, Bias, Safety & Security, and Privacy Subcards. Please report security vulnerabilities or NVIDIA AI concerns here.
License
Use of the models in this blueprint is governed by the NVIDIA AI Foundation Models Community License.
Terms of Use
GOVERNING TERMS: The software and materials are governed by the NVIDIA Software License Agreement and the Product-Specific Terms for NVIDIA AI Products, except that models are governed by the AI Foundation Models Community License Agreement and the NVIDIA RAG dataset is governed by the NVIDIA Asset License Agreement. ADDITIONAL INFORMATION: for Meta/llama-3.1-70b-instruct model the Llama 3.1 Community License Agreement, for nvidia/llama-3.2-nv-embedqa-1b-v2model the Llama 3.2 Community License Agreement, and for the nvidia/llama-3.2-nv-rerankqa-1b-v2 model the Llama 3.2 Community License Agreement. Built with Llama.
