Abstract:  Combining virtual and real visual elements into a single, realistic image requires the accurate estimation of the lighting conditions of the real scene. In recent years, several approaches of increasing complexity—ranging from simple encoder-decoder architecture to more sophisticated volumetric neural rendering—have been proposed. While the quality of automatic estimates has increased, they have the unfortunate downside of being hard to understand: they do not lend themselves to being easily editable by a user. This quickly becomes problematic when erroneous automatic results need to be corrected for improved accuracy or when creative freedom is required. In this talk, I will present recent efforts in designing editable lighting representations that 1) disentangle various components of illumination; 2) allow an intuitive control over those components; and, of course, 3) enable realistic relighting results. I will show that using our illumination estimates for applications like 3D object insertion can achieve photo-realistic results on a wide variety of challenging scenarios.

Bio: Jean-François Lalonde, Ph.D., is Full Professor in the Electrical and Computer Engineering Department at Université Laval. Prior to joining U. Laval in 2013, he was a Post-Doctoral Associate at Disney Research, Pittsburgh. He received a Ph.D. in Robotics from Carnegie Mellon University in 2011. His Ph.D. thesis won the CMU School of Computer Science Distinguished Dissertation Award. His research interests lie at the intersection of computer vision, computer graphics, and machine learning. In particular, he is interested in exploring how physics-based models and data-driven machine learning techniques can be unified to invert the image formation process and recover scene properties such as lighting, shape and reflectance. To this end, his group has captured and published the largest datasets of indoor and outdoor high dynamic range images, freely available for research. He is actively involved in bringing research ideas to commercial products, as demonstrated by his patents and technology transfers with large companies such as Adobe, and involvement as scientific advisor for high tech startups.

Homepage:  www.jflalonde.ca

Sponsored in part by:   Facebook Reality Labs Pittsburgh