Abstract:
Manipulation is a key challenge in the robotic fields that impedes the deployment of robots in real-world scenarios. While notable advancements have been made in solving high/mid level planning problems, such as decomposing tasks (e.g. “bring me a bottle”) into primitives (e.g. “pick up bottle”), the acquisition of fundamental manipulation primitives remains a difficult obstacle. This thesis aims to address this challenge by developing learning algorithms capable of acquiring primitive manipulation behaviors that are often taken for granted by humans: these tasks include picking, pushing, opening, etc. To ensure the practicality of these systems in real-world applications, where robot data is both scarce and costly, this thesis explores methods for learning from diverse offline datasets. The initial investigation focuses on the capacity of high-dimensional neural networks to learn and imitate expert behaviors, utilizing a straightforward algorithmic framework. Subsequently, we demonstrate the effectiveness of learning dynamics models from extensive offline robot data, enabling rapid adaptation to novel environments and problem-solving in new tasks. In the third work, we propose a novel approach to bootstrap an imitation controller and a dynamics model from pre-trained visual representations obtained through human videos, thereby enabling faster robotic imitation. Throughout the research process, we incorporate increasingly diverse datasets, transitioning from robot demonstrations to human videos, with the overarching objective of minimizing the dependency on robot-specific and task-specific supervision data. The final proposed work seeks to conduct a deeper analysis of the visuo-motor pretraining setup, and aims to identify the most effective data sources for learning robust and efficient robotic representations.

Thesis Committee Members:
Abhinav Gupta, Chair
Shubham Tulsiani
Deepak Pathak
Sergey Levine, UC Berkeley

More Information