Light-fields

Although the concept of the “light-field” of a scene dates back far earlier, Levoy and Hanrahan’s seminal 1996 SIGGRAPH “Light-Field Rendering” paper created a large amount of interest in the graphics community in the capture and use of the light-field for rendering. The light-field is also of great interest in the computer vision community where it has a variety of other applications besides rendering. We have explored a couple of such applications as outlined below:

Stereo

When is stereo unique and when is it inherently ambiguous? We have investigated this theoretical question and derived a concise characterization of when the stereo problem (given the entire light-field) has a unique solution and when there are multiple scenes that could have generated the same set of photometric measurements (the light-field.)

Face Recognition

We have developed an appearance-based face recognition algorithm that can operate given any subset of the light-field of the face. We call this algorithm “eigen light-fields” because it is a generalization of “eigen faces.” The training and testing subsets of the light-field do not need to overlap. Hence, our algorithm can perform face recognition across pose; for example, it can recognize a person from a profile view even though the algorithm has only seen that person from the front in the training data.