Abstract:
General purpose robots should be able to perform arbitrary manipulation tasks, and get better at performing new ones as they obtain more experience. The current paradigm in robot learning involves training a policy, in simulation or directly in the real world, with engineered rewards or demonstrations. However, for robots that need to keep learning new tasks, such policies might not be effective, since they are trained for a fixed objective.

In this talk, I will describe approaches we developed for robots to learn in a task-agnostic manner, allowing them to learn from multi-task data and continuously improve in capability.
The first involves autonomous exploration, where the robot discovers how to meaningfully interact with different objects without any task reward. The second leverages large-scale human video to learn a world model in a structured action-space, which can then be used by the robot for efficient control. We deploy both approaches on real robot systems, and observe that the robots can learn to perform tasks of interest within 30min – 1hr of interaction time across a diverse set of manipulation settings.

Committee:
Deepak Pathak
Abhinav Gupta
Deva Ramanan
Sudeep Dasari