Monocular Depth Reconstruction using Geometry and Deep Convolutional Networks

Abstract

In this thesis, we explore methods of building dense depth map from monocular video using geometry and deep convolutional networks. We focus on a particular challenging case: small motion videos, in which depth error grows large as camera movement reduces.

First, we introduce our multiview stereo pipeline. Our pipeline consists of four stages: point tracking, bundle adjustment, photometric bundle adjustment, and densification. Here we demonstrate that using photometric bundle adjustment helps in getting accurate depth of textured regions from small motion video.

The traditional multiview stereo approaches rely on heuristic local smoothness priors for low-texture regions. We improve the depth estimation of low-texture region by fusing deep convolutional network predictions. We categorize the depth fusion methods into two categories: late integration and early integration. Late integration uses highly confident partial depth from pure geometric methods as anchor points to refine the dense depth map generated by deep convolutional networks. However in this case, the network output is not guaranteed to be aligned with confident partial depth, thus the fusion process might be problematic. To improve this issue, we propose early integration, which uses confident partial depths as constraints for deep convolutional networks. This method ensures the two depth sources to be well-aligned and thus has better depth accuracy than previous methods.