Abstract: Generative models can produce compelling pictures of realistic scenes. Objects are in sensible places, surfaces have rich textures, illumination effects appear accurate, and the models are controllable. These models, such as StyleGAN, can also generate semantically meaningful edits of scenes by modifying internal parameters. But do these models manipulate a purely abstract representation of the world, or do they work in terms of familiar rendering variables?

In this talk, I will first present our method, StyLitGAN, which prompts StyleGAN to generate scenes with new lighting conditions. StyLitGAN generates scenes with realistic lighting effects, including cast shadows, soft shadows, inter-reflections, and glossy effects, without the need for labeled, paired, or CGI data. I’ll also show a near-perfect GAN inversion technique, Make It So, that outperforms previous SOTA GAN inversion methods by order of magnitude, able to invert and relight real scenes, even never seen out-of-domain images. Lastly, I’ll present strong evidence that StyleGAN has easily accessible and accurate internal representations of familiar scene properties like normals and depth. I will conclude by discussing their exciting implications for Generative AI.

Bio: Anand Bhattad is a PhD candidate working with David Forsyth at UIUC. His research interest lies at the intersection of computer vision, computational photography, computer graphics, and machine learning. His current research focuses on neural rendering and image-based lighting. His recent work, DIVeR, received the best paper nomination at CVPR 2022. He was awarded outstanding emergency reviewer at CVPR 2021 and recognized as an excellent teaching assistant in 2016. More information is available on his webpage: https://anandbhattad.github.io/

Homepage:  https://anandbhattad.github.io/

Sponsored in part by:   Meta Reality Labs Pittsburgh