Abstract:
Soft robotic manipulators present many unique advantages in difficult manipulation tasks. The inherent compliance of soft robots’ constituent deformable material makes them safe and reliable in delicate tasks such as harvesting fruit and assisting in household work. To address challenges in proprioceptive and tactile sensing for soft robots, we present a family of vision-based methods that rely on cameras embedded in the soft robot’s body. For proprioceptive sensing, we present a pipeline to generate soft robot shape data in simulation and zero-shot transfer the trained model to the real world. For tactile sensing, we propose a method to reconstruct 3D textures of contacting objects with our novel soft robotic manipulator that we call PneuGelSight. Experimental results demonstrate that the high spatial resolution of the cameras enables us to capture both high degrees of freedom of soft robotic manipulators and minute tactile features.

Committee:
Prof. Wenzhen Yuan
Prof. Jean Oh
Prof. Nancy Pollard
Dominik Bauer