Organizations can benefit from using generative AI to enhance customer service but face challenges like fragmented data sources and potential data risks—this blueprint helps them address those issues. This NVIDIA AI Blueprint leverages retrieval-augmented generation (RAG) and generative AI technologies including NVIDIA NIM™ and NVIDIA NeMo™ Retriever.
It uses some of the latest AI agent-building methodologies, connecting disparate data sources to improve the operational efficiency of existing solutions or build new customer service-centric systems. It offers advanced AI tools, secure management of sensitive data wherever it resides, personalized multi-turn question answering, sentiment analysis, summary generation, and configurable session handling.
NVIDIA AI Blueprints provide customizable generative AI reference architectures designed to equip enterprise developers with essential assets such as NIM microservices, reference code, detailed documentation, and Helm charts for deployment. These blueprints serve as a foundation for building advanced AI virtual assistants, either as standalone applications or as enhancements to existing systems. Their focus is on enabling personalization, summarization, and sentiment analysis, particularly through the use of generative AI for data that’s often inaccessible.
The blueprint includes a reference UI and an AI assistant (developed using the LangGraph framework) that leverages sub-agents to handle queries from both structured and unstructured data sources.
The following NIM microservices are used in this blueprint:
The blueprint comes with synthetic sample data representing a typical customer service function, including customer profiles, order histories (structured data), and technical product manuals (unstructured data). A notebook is provided to guide users on how to ingest both structured and unstructured data efficiently.
This reference solution implements three sub-agents using the open-source LangGraph framework. These sub-agents address common customer service tasks for the included sample dataset. They rely on the Llama 3.1 models (70B and 8B Instruct) and NVIDIA NIM microservices for generating responses, converting natural language into SQL queries, and assessing the sentiment of the conversation.
To support operational requirements, the blueprint includes reference code for managing key administrative tasks:
These features ensure that customer service teams can efficiently monitor and evaluate interactions for quality and performance.
The blueprint comes with pre-built APIs that support continuous model improvement. The feedback loop, or “data flywheel,” allows LLM models to be fine-tuned over time to enhance both accuracy and cost-effectiveness. Feedback is collected at multiple points in the process to refine the models’ performance further.
In summary, this NVIDIA AI Blueprint offers a comprehensive solution for building intelligent, generative AI-powered virtual assistants for customer service, leveraging structured and unstructured data to deliver personalized and efficient support. It includes all necessary tools and guidance to deploy, monitor, and continually improve the solution in real-world environments.
The AI virtual assistant pipeline supports the following hardware:
NVIDIA NIM inference microservices
Explore example walkthroughs on the NVIDIA API catalog through the specific NIM microservices links below:
NeMo Retriever Embed QA NeMo Retriever Rerank Mistral 4B Llama 3.1 70B Instruct
You can find the complete example of how to get started with this Blueprint available on the NVIDIA GitHub repository.
The AI Virtual Assistant Customer Service Blueprint Application is shared as reference architecture and is provided “as is”. The security in the production environment is the responsibility of the end users deploying it. When deploying in a production environment, please have security experts review any potential risks and threats (including direct and indirect prompt injection); define the trust boundaries, secure the communication channels, integrate AuthN & AuthZ with appropriate access controls, keep the deployment including the containers up to date, ensure the containers are secure and free of known vulnerabilities. Only authenticated users must be able to submit queries.
NVIDIA believes Trustworthy AI is a shared responsibility, and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their supporting model team to ensure the models meet requirements for the relevant industry and use case and address unforeseen product misuse. For more detailed information on ethical considerations for the models, please see the Model Card++ Explainability, Bias, Safety & Security, and Privacy Subcards. Please report security vulnerabilities or NVIDIA AI concerns here.
Use of the models in this AI virtual assistant for customer service blueprint is governed by the NVIDIA AI Foundation Models Community License.
GOVERNING TERMS: The blueprint is governed by the NVIDIA Agreements | Enterprise Software | NVIDIA Software License Agreement and NVIDIA Agreements | Enterprise Software | Product Specific Terms for AI Product.
GOVERNING TERMS: The NIM container is governed by the NVIDIA Software License Agreement and the Product Specific Terms for AI Products;
Use of this model is governed by the NVIDIA AI Foundation Models Community License Agreement. ADDITIONAL INFORMATION: Llama 3.1 Community License Agreement, Built with Llama.
GOVERNING TERMS: The NIM container is governed by NVIDIA Agreements | Enterprise Software | NVIDIA Software License Agreement and NVIDIA Agreements | Enterprise Software | Product Specific Terms for AI Product; and the use of this model is governed by the ai-foundation-models-community-license.pdf (nvidia.com). ADDITIONAL INFORMATION: MIT license.
GOVERNING TERMS: The NIM container is governed by NVIDIA Agreements | Enterprise Software | NVIDIA Software License Agreement and NVIDIA Agreements | Enterprise Software | Product Specific Terms for AI Product; and the use of this model is governed by the ai-foundation-models-community-license.pdf (nvidia.com). ADDITIONAL INFORMATION: Apache license.
Create intelligent virtual assistants for customer service across every industry