Generate exponentially large amounts of synthetic motion trajectories for robot manipulation from just a few human demonstrations.
Imitation learning lets robots learn skills from observing human demonstrations. But gathering enough high-quality real-world datasets can be challenging, costly, and time-consuming. Synthetic data, generated from physically accurate simulations, addresses the challenge of limited real-world data acquisition by accelerating data collection and providing the diversity needed to generalize robot learning models.
The NVIDIA Isaac GR00T blueprint for synthetic manipulation motion generation is the ideal place to start. This is a reference workflow for creating exponentially large amounts of synthetic motion trajectories for robot manipulation from a small number of human demonstrations, built on NVIDIA Omniverse™ and NVIDIA Cosmos™.
First, developers use a spatial computing device such as the Apple Vision Pro to portal into their simulated robot digital twin and record motion demonstration teleoperating a simulated robot. These recordings are then used to generate a larger set of physically accurate synthetic motion trajectories. Finally, the blueprint further augments the dataset by generating an exponentially large, photorealistic, and diverse set of training data.
The overall experience is divided into four distinct parts:
1. Choose from a pre-recorded set of human demonstrations.
2. View the synthetically generated motion.
3. Select from the list of pre-populated prompts to augment the generated motions.
4. Click "View Source Code” to retrieve the blueprint from GitHub.
Input:
Output:
Hardware Requirements
GPU
CPU
NVIDIA believes trustworthy AI is a shared responsibility, and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their supporting team to ensure the technologies meet requirements for the relevant industry and use case and address unforeseen product misuse. Please report security vulnerabilities or NVIDIA AI concerns here.
Licensing information for Isaac Lab can be found here.
License information for For NVIDIA Cosmos: can be found under NVIDIA Open Model License
Licensing for GR00T-Mimic can be found here
Governing Terms: This trial service is governed by the NVIDIA API Trial Terms of Service.