Title: Introspective Perception through Identifying Blur, Light Direction, and Angle-of-View

Abstract

Robotic perception tasks have achieved great performance, especially in autonomous vehicles and robot assistance. However, we still often do not understand how and when perception tasks fail. Researchers have achieved some success in creating introspective perception systems that detect when perception tasks will fail, but they usually are tuned to only specific, connected perception tasks and do not identify the reasons for failure such as blur, light direction changes, and angle-of-view changes. To address this shortcoming, we designed a combined introspective perception system that detects three common failure types: blur, light direction, and angle-of-view. We split these failure cases into two target areas of research: Blur Detection & Classification and Light Direction & Angle-of-View Failure Prediction. We then combined these subsystems into one introspective perception system that allows us to identify failure in the perception task. Using perception for non-destructive block removal as a use case, we show that an introspective perception system can identify these three failure types to inform appropriate robot actions.

Committee:

Aaron Steinfeld (advisor)

David Held