A Generative Paradigm for Building Generalist Robots: Infrastructure, Scaling, and Policy Learning

Abstract

Robotics researchers have been attempting to extend data-driven breakthroughs in fields like computer vision and language processing into robot
learning. However, unlike vision or language domains where massive amounts of data is readily available on the internet, training robotic policies relies on physical and interactive data collected via interacting with the physical world – a resource-intensive process limited by labor constraints. Such data scarcity has long been a major bottleneck in scaling up robot learning systems, constraining prior efforts to small-scale and task-specific settings. In this thesis, we present a generative paradigm that could potentially lead to general purpose robots by addressing existing limitations. This is
achieved through three self-contained yet interdependent lines of work, which, when integrated, collectively form a cohesive and comprehensive paradigm:
• We propose building comprehensive world simulator infrastructures for modeling the physical world, both learning based and rule-based,
to create a virtual yet realistic and powerful world for robotic agents to explore and develop their skills.
• We present Generative Simulation, a generative framework for autonomously scaling up robotic data generation better leveraging
the power of compute, built on top of the world models we built. Traditional policy training in simulation has long been hindered
by extensive human effort in designing tasks, assets, environments, training supervisions, and evaluation metrics. We design a robotic
agent that automates all stages of simulated robot learning – from initial task proposal to policy training – leading to diverse robotic
demonstrations.
• We present neural network architectures and learning methods for distilling collected demonstration data into unified multi-modal robotic
policies, completing the cycle from data generation to effective policy training.