---
title: "AI Factory Operations Agent"
publisher: "nvidia"
type: "blueprint"
updated: "2026-09-24T07:14:21.888Z"
description: "Deploy AI agents to investigate cluster issues and streamline governed AI factory operations."
canonical: "https://build.nvidia.com/nvidia/operate-ai-factories-with-agents"
---

AI Factory Operations Agent is an NVIDIA blueprint for deploying an extensible agentic operations framework built with NemoClaw. Specialized agents gather and correlate evidence across cluster systems, generate clear root cause summaries, and recommend next steps through governed, auditable workflows that teams can adapt to their own AI factory environments.

## AI Factory Operations Agent Architecture

The blueprint routes operator requests through the NemoClaw agent runtime and OpenShell sandbox layer to approved SRE tools for Kubernetes, Slurm, cluster management, observability, GPU telemetry, hardware diagnostics, and research-assisted root cause analysis (RCA).

![AI factory agent architecture](https://assets.ngc.nvidia.com/products/api-catalog/operate-ai-factories-with-agents/diagram.jpg)

## Key Features:

- **Agent chat interface:** Interact with specialized agents through a pre-built chat interface, or connect with your existing AI tool of choice, for investigations and cluster health checks.
- **Cluster health summaries:** Get alerts for cluster health status, workload anomalies, and recurring issue patterns.
- **Root cause analysis (RCA):** Identify issues faster with extensible SRE skills.
- Use domain-specific agents that autonomously search across scheduler logs, Kubernetes state, node health, and GPU signals to identify reasons for job interruptions.
- Optimize cluster operational health with research and runbook-assisted RCA using connected documentation, operational procedures, and site-specific knowledge sources
- **Cluster management:** Quickly request node states, schedulability, inventory, and health investigation from NVIDIA Base Command Manager or NVIDIA Mission Control.
- **Observability and monitoring:** Generate dashboards using Prometheus, Grafana, DCGM GPU telemetry, and more to get insights quicker.
- **Customizable:** Bring your own model and APIs to customize the agents for your Ai factory deployment.
- Use NVIDIA Nemotron or bring your own model (most major models supported.)
- Connect APIs from additional tools to add new agents and workflows
- **Governance:** Manage controlled operations through read-only mode and NVIDIA NemoClaw.

## How the Blueprint Works

An SRE or operations team asks the system to investigate a cluster issue, summarize health, or explain a workload failure. The runtime reasons over the request, routes it to the right domain-specific agents or skills, and uses approved tools through governed OpenShell execution. The agents gather evidence from systems such as Kubernetes, Slurm, Prometheus, Grafana, NVIDIA Base Command Manager, and hardware diagnostics, then correlate the findings into a clear root cause analysis summary with recommended next steps. Operator feedback and repeated workflows can be captured to help teams standardize their preferred way of running AI factory operations over time.

## Minimum System Requirements

- A Kubernetes cluster with Helm 3 and accessed with kubectl (installs on most existing AI factory control planes).
- Persistent storage for runtime state and audit logs.
- Bring your own LLM: Connect to most open model inference endpoints, either cloud-hosted or on-premises. On-prem will require additional accelerated compute to run the model.
- Optional integrations require their own endpoints or access (e.g., Prometheus/Grafana for observability, Slurm evidence or log access for job RCA, NVIDIA Mission Control or NVIDIA Base Command Manager access for cluster management, and site-specific runbooks for customized RCA.)

## Get Started

Visit [GitHub](https://github.com/NVIDIA/AI-Factory-Operations-Agent) to download the blueprint.