Event Location: NSH 3305

Abstract: Endoscopy is attracting increasing attention for its role in minimally invasive, computer-assisted and tele-surgery. Analyzing images from endoscopes to obtain meaningful information about anatomical structures such as their 3D shapes, deformations and appearances, is crucial to such surgical applications. However, 3D reconstruction from endoscopic images is challenging due to the small field of view of the endoscope, large image distortion, featureless bones and occluding and messy blood. In this thesis, a novel methodology is developed for accurate 3D bone reconstruction from endoscopic images, by exploiting and enhancing computer vision techniques such as shape from shading, tracking and statistical modeling. Second, an efficient and robust approach is proposed to track soft tissues to obtain 3D visualizations of the medically relevant anatomical features.

We first design a complete calibration scheme to estimate both geometric and photometric parameters including the rotation angle, light intensity and light sources’ spatial distribution. This is crucial to our further analyses of endoscopic images. For hard tissues (bones), a solution is presented to reconstruct the Lambertian surface of bones using a sequence of overlapped endoscopic images, with only partial boundaries visible in each image. We extend the classical shape-from-shading approach to deal with perspective projection and near point light sources that are not co-located with the camera center. Then, by tracking the endoscope, the complete occluding boundaries of the bone is obtained by aligning the partial boundaries from different images. Finally a complete and consistent shape is obtained by simultaneously growing the surface normals and depths in all views. In order to deal with the over-smoothness and occlusions, we employ a statistical atlas to constrain and refine the mutli-view shape from shading. A two-level framework is also developed for efficient atlas construction.

For soft tissues that have with richer color and texture, we obtain 3D views by using Telestration markers and matching. For this, we developed a real-time 3D Telestration system to locate and track the correspondences between a pair of stereo images from an endoscope with two cameras. We implemented a hierarchical feature descriptor to represent the Telestration markers which are selected by surgeons. Our feature descriptor is robust to environmental noise: a global geometric constraint is discovered to deal with occlusions caused by smoke and tool interactions. Finally, the correspondences are used to provide 3D view of Telestration by fusing two images in the da Vinci surgical system. We believe this thesis will help toward broader application of endoscopes during surgery.

Committee:
Branislav Jaramaz, Co-chair

Srinivasa Narasimhan, Co-chair

Yanxi Liu

Ko Nishino, Drexel University