Stereo Matching with Reflections and Translucency - Robotics Institute Carnegie Mellon University

Stereo Matching with Reflections and Translucency

Yanghai Tsin, Sing Bing Kang, and Richard Szeliski
Conference Paper, Proceedings of (CVPR) Computer Vision and Pattern Recognition, pp. 702 - 709, June, 2003

Abstract

In this paper, we address the stereo matching problem in the presence of reflections and translucency, where image formation can be modeled as the additive superposition of layers at different depth. The presence of such effects violates the Lambertian assumption underlying traditional stereo vision algorithms, making it impossible to recover component depths using direct color matching based methods. We develop several techniques to estimate both depths and colors of the component layers. Depth hypotheses are enumerated in pairs, one from each layer, in a nested plane sweep. For each pair of depth hypotheses, we compute a component-color-independent matching error per pixel, using a spatial-temporal-differencing technique. We then use graph cut optimization to solve for the depths of both layers. This is followed by an iterative color update algorithm whose convergence is proven in our paper. We show convincing results of depth and color estimates for both synthetic and real image sequences.

BibTeX

@conference{Tsin-2003-8665,
author = {Yanghai Tsin and Sing Bing Kang and Richard Szeliski},
title = {Stereo Matching with Reflections and Translucency},
booktitle = {Proceedings of (CVPR) Computer Vision and Pattern Recognition},
year = {2003},
month = {June},
pages = {702 - 709},
keywords = {stereo matching, reflection, translucency},
}