Title: Internet Supervision for Robot Learning

Abstract: The availability of internet-scale data has led to impressive large-scale AI models in various domains, such as vision and language. For learning robot skills, despite recent efforts in crowd-sourcing robot data, robot-specific datasets remain orders of magnitude smaller. Rather than focusing on scaling robot data, my research takes the alternative path of directly using available internet data and models as supervision for robots — in particular, learning general feedback models for robot actions. Feedback can be relatively agnostic to robot embodiments, applicable to various policy learning algorithms, and as I will show, can be learned even from exclusively non-robot data. I will present two complementary approaches in this talk. First, I will present a novel reinforcement learning algorithm that can directly use in-the-wild human videos to learn value functions, producing zero-shot dense rewards for manipulation tasks specified in images and texts. Second, I will demonstrate how grounding large language models code search with simulator feedback enables automated reward design for sim-to-real transfer of complex robot skills, such as a quadruped robot dog balancing on a yoga ball.

Bio: Jason Ma is a fifth-year PhD student at the University of Pennsylvania. His research interests span robot learning, reinforcement learning, and deep learning. His work has received Best Paper Finalist at ICRA 2024, Top 10 NVIDIA Research Projects of 2023, and covered by popular media such as the Economist, Fox, Yahoo, and TechCrunch. Jason is supported by Apple Scholar in AI/ML PhD Fellowship as well as OpenAI Superalignment Fellowship.