A Multi-Image Shape-from-Shading Framework for Near-Lighting Perspective Endoscopes

Abstract

This article formulates a near-lighting shape-from-shading problem with a pinhole camera (perspective projection) and presents a solution to reconstruct the Lambertian surface of bones using a sequence of overlapped endoscopic images, with partial boundaries in each image. First we extend the shape-from-shading problem to deal with perspective projection and near point light sources that are not co-located with the camera center. Secondly we propose a multi-image framework which can align partial shapes obtained from different images in the world coordinates by tracking the endoscope. An iterative closest point (ICP) algorithm is used to improve the matching and recover complete occluding boundaries of the bone. Finally, a complete and consistent shape is obtained by simultaneously regrowing the surface normals and depths in all views. In order to fulfill our shape-from-shading algorithm, we also calibrate both geometry and photometry for an oblique-viewing endoscope that are not well addressed before in the previous literatures. We demonstrate the accuracy of our technique using simulations and experiments with artificial bones.