Abstract:
To integrate robots seamlessly into daily life, they must be able to handle a variety of tasks in diverse environments, like cooking in restaurants or tidying up around the house. Many of the items in these environments are deformable such as fruits or bed sheets and a certain level of dexterity is necessary to accomplish complex tasks such as making breakfast or folding clothes. However, existing industrial robotic grippers have limitations in terms of sensing and dexterity, and anthropomorphic hands are expensive and challenging to control. This thesis aims to address the challenges of dexterously manipulating deformable objects by presenting work on low-cost multimodal sensing and a novel end-effector design with high dexterity but simple control.

The first part of this thesis focuses on low-cost multimodal sensing. First, we will discuss the abilities of vibrotactile sensing to distinguish material properties between different food items in a self-supervised manner.
Afterwards, we introduce work on characterizing the performance of vibrotactile sensing in a factory environment where there are tight tolerances when inserting industrial connectors. Finally, we demonstrate how magnetic sensing can help a robot localize different items such as a key and sense forces less than 10N when manipulating deformable tools such as a sauce dispensing bottle.

In the second part of this thesis, I will introduce our work on utilizing mini-delta robots as robotic fingers that are both dexterous, and simple to control. With three translational degrees of freedom each, we were able to teleoperate a two fingered DeltaHand to perform various dexterous tasks like card picking, grape plucking, and dough rolling. We also developed a more compact DeltaHand with four fingers where the additional fingers help stabilize objects during in-hand tasks and enable the hand to fold cloth, unscrew a cap, and straighten a cable.

Finally, to bring both parts of the thesis together, I will present work on utilizing different sensors to learn dexterous closed loop skills on a DeltaHand using various multimodal skills on a variety of tasks with fine-grained demonstrations. In addition, we augment a parallel jaw gripper with a Delta finger in order to perform tasks that require both force from the parallel jaw fingers as well as dexterity from the Delta finger to succeed such as picking up a soft tortilla, removing a stick of gum from a pack, and lifting up the cap of a vitamin bottle. In summary, this thesis seeks to advance robotic capabilities in handling diverse objects and tasks through innovative sensing methods, dexterous manipulation, and improved control policies.

Thesis Committee Members:
Oliver Kroemer (Co-Chair)
Manuela Veloso (Co-Chair)
Zeynep Temel
Tapomayukh Bhattacharjee (Cornell University)

More Information