Abstract: Video communication connects our world. It is necessary in conducting business, educational and personal activities across different geographical locations. However, the quality of an average user’s video communication is dramatically worse than that of professionally created videos in news broadcasts, talk shows, and on YouTube. This is because professionally created videos are often captured with high-quality cameras and lighting set-ups in a studio environment and edited with software requiring human expertise (e.g. Adobe After Effects), all of which are not accessible to regular content creators or video callers. Effective video communication requires high visual quality, real-time performance, and an interface that enhances interaction, which is absent in our current systems. In this talk I will discuss my efforts in creating next-gen video communication systems – as close as possible to real-life interactions. I use Computer Vision & Graphics to build equitable and accessible video production tools, requiring minimal human expertise to produce high-quality videos in real-time, thus democratizing VFX.  First, I will discuss my efforts in producing real-time high-resolution background replacement without requiring a Green Screen studio. Next, I will discuss my ongoing research in creating personalized AI models from the imagery captured with a webcam. Specifically, I will discuss my effort in building personalized relighting models to improve lighting during a video call without requiring expensive hardware setups like the Light Stage. This part of the talk will also highlight how lighting variation can also enable 3D reconstruction and subsequently improve video communication with Augmented Reality.

Bio: Soumyadip Sengupta is a Postdoctoral Research Associate in Paul G. Allen School of Computer Science and Engineering at the University of Washington, Seattle. He is affiliated with GRAIL (Graphics and Imaging Laboratory), working with Steve Seitz, Brian Curless, and Ira Kemelmacher-Shlizerman. He received his Ph.D. at the University of Maryland, College Park in 2019 advised by David Jacobs. His research interest lies at the intersection of Computer Vision, Computational Photography, and Computer Graphics, with a focus on democratizing visual effects (VFX) to improve video communication. His work on Background Matting received Best Student Paper Honorable Mentions at CVPR 2021 and has been adopted by various companies, e.g. Microsoft, Inter State Studio, LightTwist etc.

Homepage:  https://homes.cs.washington.edu/~soumya91/

Sponsored in part by:   Facebook Reality Labs Pittsburgh