Generalizable Learning-based Registration

Abstract

This thesis explores the application of deep learning algorithms to 2D (image) and 3D (point cloud) registration scenarios, especially where the challenging nature of the data precludes the use of classical methods for establishing correspondence and alignment.

In the 2D case, we apply a recently-proposed learning-based image registration method to the problem of aligning outdoor imagery taken across seasons and times of day. Further, we extend the method to perform GPS-denied UAV geolocalization by aligning UAV images and satellite imagery. We also propose a novel joint optimization of motion estimates from visual odometry and geolocalization, to increase localization accuracy in cases where parts of the satellite map may be missing or too dissimilar from the UAV imagery.

In the 3D case, we develop three novel point cloud registration algorithms based on state-of-the-art point cloud processing deep networks. We show the benefits of the learned representation for registration on partial data, intra-category data (same category, different shape), and real-world data.