3D Reconstruction of Anatomical Structures from Endoscopic Images

Abstract

Endoscopy is attracting increasing attention for its role in minimally invasive, computer-assisted and tele-surgery. Analyzing images from endoscopes to obtain meaningful information about anatomical structures such as their 3D shapes, deformations and appearances, is crucial to such surgical applications. However, 3D reconstruction of bones from endoscopic images is challenging due to the small field of view of the endoscope, large image distortion, featureless surfaces and occlusion by blood and particles. In this thesis, a novel methodology is developed for accurate 3D bone reconstruction from endoscopic images, by exploiting and enhancing computer vision techniques such as shape from shading, tracking and statistical modeling. We first designed a complete calibration scheme to estimate both geometric and photometric parameters including the rotation angle, light intensity and light sources’ spatial distribution. This is crucial to our further analysis of endoscopic images. A solution is presented to reconstruct the Lambertian surface of bones using a sequence of overlapped endoscopic images, where only partial boundaries are visible in each image. We extend the classical shape-from-shading approach to deal with perspective projection and near point light sources that are not co-located with the camera center. Then, by tracking the endoscope, the complete occluding boundary of the bone is obtained by aligning the partial boundaries from different images. A complete and consistent shape is obtained by simultaneously growing the surface normals and depths in all views. Finally, in order to deal with over-smoothness and occlusions, we employ a statistical atlas to constrain and refine the multi-view shape from shading. A two-level framework is also developed for efficient atlas construction.