Abstract: In this talk, I will present three experiments that use low-level image statistics to generate high-resolution detailed outputs. In the first experiment, I will use 2D pixels to efficiently mine hard examples for better learning. Simply biasing ray sampling towards hard ray examples enables learning of neural fields with more accurate high-frequency detail in less time. The second experiment leverages 2D pixels to learn a denoising model from the collection of images. This denoising model enables detailed high-frequency outputs from the model trained on low-resolution samples. The final experiment builds a representation of a pixel that contains color and depth information accumulated from multi-views for a particular location and time along a line of sight. This pixel-based representation alongside a multi-layer perceptron allows us to synthesize novel views given a discrete set of multi-view observations as input. The proposed formulation reliably operates on sparse and wide-baseline multi-view images/videos and can be trained efficiently within a few seconds to 10 minutes for hi-res (12MP) content.

Bio: Aayush Bansal received his Ph.D. in Robotics from Carnegie Mellon University under the supervision of Prof. Deva Ramanan and Prof. Yaser Sheikh. He was a Presidential Fellow at CMU, and a recipient of the Uber Presidential Fellowship (2016-17), Qualcomm Fellowship (2017-18), and Snap Fellowship (2019-20). His research has been covered by various national and international media such as NBC, CBS, WQED, 90.5 WESA FM, France TV, and Journalist. He has also worked with production houses such as  BBC Studios, Full Frontal with Samantha Bee (TBS), etc.

Homepage: https://www.aayushbansal.xyz/

Sponsored in part by:   Meta Reality Labs Pittsburgh