Abstract: Burst imaging pipelines allow cellphones to compensate for less-than-ideal optical and sensor hardware by computationally merging multiple lower-quality images into a single high-quality output. The main challenge for these pipelines is compensating for pixel motion, estimating how to align and merge measurements across time while the user’s natural hand tremor involuntarily shakes the camera. In this work, we explore continuous projective models of burst photography, backed by multi-resolution neural field representations, and fit to real in-the-wild mobile burst captures. These task-specific models not only estimate and compensate for pixel motion, but use it as a powerful source of geometric information to estimate scene depth, see behind occlusions, separate reflections, and erase photographer-cast shadows.

BIO: Ilya Chugunov is a PhD candidate in the Princeton Computational Imaging Lab, advised by Professor Felix Heide. His work focuses on neural field representations for inverse imaging problems, depth reconstruction, and computational photography. He received his bachelor’s in electrical engineering and computer science from UC Berkeley, where he worked on low-rank reconstruction methods for magnetic resonance imaging with Professors Moriel Vandsburger and Miki Lustig. Ilya is an NSF graduate research fellow and, when not in the office, an amateur nature photographer.

Homepage: https://ilyac.info/

Sponsored in part by:   Meta Reality Labs Pittsburgh