Abstract: Understanding the 3D structure of real-world environments is a fundamental challenge in machine perception, critical for applications spanning robotic navigation, content creation, and mixed reality scenarios. In recent years, machine learning has undergone rapid advancements; however, in the 3D domain, such data-driven learning is often very challenging under limited 3D/4D data availability. In this talk, we first explore learning 3D priors from data capture and annotation for supervision, leveraging synthetic data as a strong 3D prior for reconstruction and semantic understanding of 3D scenes observed from commodity RGB and RGB-D sensors. As synthetic priors can be limited in diversity, we then discuss real-world 3D data alternatives, followed by relaxing 3D supervision constraints to weakly supervised formulations for such object-based reconstruction and 3D semantic scene understanding. Finally, as real-world scenes are often dynamic, we characterize 3D interactions and propose to distill knowledge from other data modalities to enable zero-shot 3D interaction synthesis. These 3D learning strategies promise to usher in a new paradigm of generalized 3D perception, beyond the limits of existing 3D datasets, to enable in-the-wild 3D analysis of environments.

Bio:  Angela Dai is an Associate Professor at the Technical University of Munich where she leads the 3D AI lab. Angela’s research focuses on understanding how the 3D world around us can be modeled and semantically understood. Previously, she received her PhD in computer science from Stanford in 2018, advised by Pat Hanrahan, and her BSE in computer science from Princeton in 2013. Her research has been recognized through an ERC Starting Grant, Eurographics Young Researcher Award, Google Research Scholar Award, ZDB Junior Research Group Award, an ACM SIGGRAPH Outstanding Doctoral Dissertation Honorable Mention, as well as a Stanford Graduate Fellowship.

Webpage:  Dai_Angela (tum.de)