Talk: Closing the perception-action loop

Abstract:

Robots and autonomous systems have been playing a significant role in the modern economy. Custom-built robots have remarkably improved productivity, operational safety, and product quality. However, these robots are usually programmed for specific tasks in well-controlled environments, unable to perform diverse tasks in the real world. In this talk, I will present my work on building more effective and generalizable robot intelligence by closing the perception-action loop. I will discuss my research that establishes a tighter coupling between perception and action at three levels of abstraction: 1) learning primitive motor skills from raw sensory data, 2) sharing knowledge between sequential tasks in visual environments, and 3) learning hierarchical task structures from video demonstrations.

Bio:

Yuke Zhu is a final year Ph.D. candidate in the Department of Computer Science at Stanford University, advised by Prof. Fei-Fei Li and Prof. Silvio Savarese. His research interests lie at the intersection of robotics, computer vision, and machine learning. His work builds machine learning and perception algorithms for general-purpose robots. He received a Master’s degree from Stanford University and dual Bachelor’s degrees from Zhejiang University and Simon Fraser University. He also collaborated with research labs including Snap Research, Allen Institute for Artificial Intelligence, and DeepMind.