A Paraperspective Factorization Method for Shape and Motion Recovery

Abstract

In this paper, we present a method for recovering both the shape of an object and its motion relative to the camera from a sequence of images of the object, using feature points tracked throughout the sequence. The method is based on the factorization method developed by Tomasi & Kanade, which achieves its accuracy by using a large number of points and images, and robustly applying a well-understood matrix computation, the singular value decomposition, to the highly redundant input data. While the Tomasi & Kanade method was based on orthographic projection, our method uses a projection model known as paraperspective projection. Orthographic projection does not account for the apparent change in size of an object as it moves toward or away from the camera, nor the different angle from which an object is viewed as it moves parallel to the image plane. In contrast, paraperspective projection closely approximates perspective projection by modelling both of these effects. A new formulation based on this projection model allows us to apply the factorization method to a wider range of scenarios, and to recover the distance from the camera to the object. The method assumes no model of the motion or of the object's shape, and recovers the shape and motion accurately even for distant objects. We present several experiments which illustrate the method's performance over a wide range of noise values and a wide range of distances from the object.