Title: Distilling View-conditioned Diffusion for 3D Reconstruction

Abstract:
We propose a 3D neural mode-seeking formulation that combines probabilistic generation of unseen regions with faithful reprojection of seen regions in a consistent 3D representation. Feature reprojection methods (NerFormer, PixelNeRF) are 3D consistent, but fail to hallucinate unseen regions. Image generation methods (ViewFormer) generate plausible hallucinations, but generated views are not 3D consistent. We generate realistic and consistent 3D using just two images.

Committee:
Prof. Shubham Tulsiani (advisor)
Prof. Jun-Yan Zhu