Abstract:
Robots operating in the real world can leverage intentional contacts with objects to understand and manipulate them effectively—especially in cluttered, partially observable environments where vision alone is insufficient. This thesis explores how intentional physical interactions, combined with haptic sensing, can provide rich spatial, temporal, and physical cues that enhance a robot’s perception and decision-making. By leveraging multimodal signals, particularly vibration-based audio from contact, we develop methods for localizing contact, inferring material properties, and improving manipulation strategies. These ideas are unified through a multimodal sensing used to close-the-loop for high-level planning and low-level policies, enabling robots to interact more intelligently through touch. Ultimately, this work aims to shift robotic manipulation toward systems that do not avoid contact, but learn from it.

Thesis Committee Members:
Oliver Kroemer (Chair)
George Kantor
Yonatan Bisk
Tapomayukh Bhattacharjee (Cornell University)

More Information