DPLVO: Direct Point-Line Monocular Visual Odometry

Abstract

In this paper, we present a direct visual odometry (VO) using points and lines. Direct methods generally choose pixels with sufficient gradients to minimize the photometric error for the status estimation. Pixels on lines are generally involved in this process. But the collinear constraint among these points are generally ignored, which may result in less accurate depth estimation. This paper introduces the collinear constraint into the state-of-the-art direct visual odometry DSO [1] to overcome this problem. The 3D lines, points and poses within a sliding window are jointly optimized. DSO implicitly establishes the data association for points among the keyframes within a sliding window by direct image alignment. This scheme is typically suitable for points, as points are generally only visible within a short time window. However, as lines are unbounded entities, they can be observed by a camera significantly longer than points. Thus, we seek to establish the long-term data association for lines among the keyframes. The 3D collinear points that are removed from the sliding window are served as collinear priors for the following windowed optimization. We prove that the prior collinear constraints of a 3D line can be compressed into six residuals in the optimization. This significantly reduces the computational complexity, and enables real-time performance for incorporating long 3D line segments into the windowed optimization. We present a new 3D line representation which reduces the four degrees of freedom (DoF) of a 3D line into two DoF by parameterizing the 3D line in the back-projection plane of its first 2D line observation. The experimental results show that our algorithm outperforms the state-of-the-art direct monocular VO algorithms.