Abstract:

With recent developments in both commodity range sensors as well as mixed reality devices, capturing and creating 3D models of the world around us has become increasingly important. As the world around us lives in a three-dimensional space, such 3D models will not only facilitate capture and display for content creation but also provide a basis for fundamental scene understanding, from semantic understanding to virtual interactions, which must be formulated in 3D for many applications such as augmented or virtual reality.

My talk will focus on constructing a generative formulation for the construction of 3D models from RGB-D data, and the use of 3D reconstructions towards a 3D semantic and instance understanding of these scenes. I will begin with the capture and reconstruction of RGB-D scan data, and discuss two major challenges towards democratization of 3D content: inferring 3D semantic understanding (by way of semantic and instance segmentation), and generating high-quality 3D model geometry. I will present 3D deep learning approaches based on 3D, RGB-D nature of these scans, to predict spatially consistent semantic and instance segmentations of 3D scenes, and follow by presenting a generative deep learning formulation, conditional on an input scan, to construct the complete geometry of a partially observed scene. In particular, this formulation enables the inference of complete 3D model geometry from partial 3D scan observations at scale, generating room- and building floor-scale models, and further extending to real-world scans.

Bio:

Angela Dai is a postdoctoral fellow at the Technical University of Munich. Her research focuses on creating high-quality 3D models of real-world environments, towards enabling human-level scene understanding and democratizing 3D scanning for content creation and mixed reality scenarios. She completed her Ph.D. in Computer Science at Stanford University, advised by Pat Hanrahan. During her PhD, she has advanced real-time 3D reconstruction, and leveraged this towards developing machine learning approaches towards improving the reconstruction quality and semantic and instance understanding of these 3D scans. Angela received her Bachelors degree in Computer Science from Princeton University. Her work has been recognized with a Professor Michael J. Flynn Stanford Graduate Fellowship and a 1.25mil euro ZDB junior research group award