Abstract:
Dexterity, the ability to perform complex interactions with the physical world, is at the core of robotics. However, existing research in robot manipulation has been focused on tasks that involve limited dexterity, such as pick-and-place. The motor skills of the robots are often quasi-static, have a predefined or limited sequence of contact events, and involve restricted object motions. In contrast, humans interact with their surroundings with dynamic and contact-rich manipulation skills, allowing us to perform a wider variety of tasks in a broader range of settings.

This thesis explores using Reinforcement Learning (RL) to equip robots with generalizable dexterity. RL has shown remarkable success in solving sequential decision-making problems, making it a promising technique for developing advanced manipulation skills. In this thesis, we examine three key challenges when applying RL to manipulation and discuss our approaches to overcome them.

First, reinforcement learning is a data-driven method, but collecting robot data is expensive and time-consuming. To reuse robot data effectively, we propose an offline RL algorithm to train a policy on a static dataset without requiring additional data collection. In addition, we discuss a framework that effectively reuses robot data across environments with non-stationary dynamics.

Second, dexterity is often assumed to be limited by the hardware design of the robot. We propose to enhance the robot’s dexterity beyond its hardware limitations by exploiting the external environment, which shows dynamic and contact-rich emergent behaviors.

Third, learning complex skills that can generalize is challenging. We propose an RL framework with an action representation that is spatially-grounded and temporally-abstracted which allows the robot to learn complex interactions that can generalize to unseen objects.

Thesis Committee Members:
David Held, Chair
Abhinav Gupta
Oliver Kroemer
Vincent Vanhoucke, Google

More Information