Vision-Based Navigation and Deep-Learning Explanation for Autonomy

Abstract

In this thesis, we investigate vision-based techniques to support robot mobile autonomy in human environments, including also understanding the important image features with respect to a classification task. Given this wide goal of transparent vision-based autonomy, the work proceeds along three main fronts. Our first algorithm enables a UAV to visually localize and navigate with respect to CoBot, a ground mobile robot, in order to perform visual search tasks. Our approach leverages the robust localization and navigation capabilities of CoBot while allowing the UAV to search for the object of interest in locations that CoBot cannot access. Second, to enable safe UAV navigation using its monocular camera, we contribute a deep learning based perception system to avoid obstacles in real-time. We demonstrate that using our system, UAVs can navigate safely in various challenging environments. Finally, we address our goal towards justification of vision-based decisions. We investigate an explanation technique to understand the predictions of a deep learning based image classifier. We contribute the Automatic Patch Pattern Labeling for Explanation (APPLE) algorithm for analyzing a deep network to find neurons that are `important' to the network classification outcome, and for automatically labeling the patches of the input image that activate these important neurons. We investigate several measures of importance for neurons and demonstrate that our technique can be used to gain insight into how a network decomposes an image to make its classification. The performance of each of these contributions is demonstrated through experimental results.