Abstract:
To integrate robots seamlessly into daily life, they must be able to handle a variety of tasks in diverse environments, like assisting in hospitals or cooking in kitchens. Many of the items in these environments are deformable such as bedding in hospitals or vegetables in kitchens, and a certain level of dexterity is necessary to accomplish complex tasks such as folding cloth or rolling dough. However, existing industrial robotic grippers have limitations in terms of sensing and dexterity, and anthropomorphic hands are expensive and challenging to control. In this thesis, we focus on addressing the challenges of dexterously manipulating deformable objects by presenting work on low-cost multimodal sensing and a novel end-effector design with high dexterity but simple control.

In this talk, I will first discuss my work on leveraging low-cost contact microphones to recognize contact changes and material properties of food items that are not visually evident, which allows the robot to adapt its cutting motion to different and novel food items. Subsequently, we are able to use these contact microphones to learn self-supervised embeddings for classifying material properties through interactive exploration. Next, we incorporate a magnetometer into an industrial robotic gripper and use deformable magnetic stickers placed on objects and tools in the environment for localization and force sensing. This approach enables precise insertion of a key into a lock and measured pipetting of water drops. Afterwards, I will introduce a low-cost dexterous manipulator using delta robots as dexterous fingers which can complete various tasks such as card picking, grape plucking, and dough rolling through human teleoperation. Finally, I will talk about my proposed work on simplifying the teleoperation process for collecting human demonstrations on these dexterous manipulators and learning closed-loop multimodal control policies to complete more complex tasks autonomously.

Thesis Committee Members:
Oliver Kroemer, Co-chair
Manuela Veloso, Co-chair
Zeynep Temel
Tapomayukh Bhattacharjee, Cornell University

More Information