Abstract:
Decades of research on computer vision has highlighted the importance of active sensing — where the agent actively controls parameters of the sensor to improve perception. Research on active perception the context of robotic manipulation has demonstrated many novel and robust sensing strategies involving a multitude of sensors like RGB and RGBD cameras, a variety of tactile, proximity and spectroscopic sensors to construct a representation of and localize itself in the robots’ workspace, estimate motion of the agent in the workspace and estimate states of objects in the robot’s workspace. In this work we look at three components of visual sensing in the context of robotic manipulation — namely the sensor configuration, sensor position and robot workspace illumination to improve perception of objects. We begin by demonstrating a robot manipulator mounted ensemble of RGB, RGBD and tactile sensors for visually servoing and accurately localizing objects using vision and touch. Following this, we show that controlling the illumination of a robot’s workspace can yield important information about the surface of the objects. This information can then be used for common manipulation tasks of estimating deformation, executing immobilizing grasps, and estimating object poses. Finally, we propose a robot mounted multi-flash stereo camera device to acquire multi-illumination, multi-view images of objects in a robot’s workspace to generate 3D visual representations of objects.

Thesis Committee Members:
Christopher G. Atkeson, Chair
Srinivasa G. Narasimhan
Oliver Kroemer
Christoph Lassner, Epic Games
Andy Zeng, Google Brain

More Information