Abstract:  Abstract: The world underlying images and videos is 3-dimensional and dynamic, i.e. 4D, with people interacting with each other, objects, and the underlying scene. Even in videos of a static scene, there is always the camera moving about in the 4D world. Accurately recovering this information is essential for building systems that can reason about and interact with the underlying scene, and has immediate applications in visual effects and creation of immersive digital worlds. However, disentangling this 4D world from a video is a particularly ill-posed inverse problem rife with fundamental ambiguities.

In this talk, I will discuss recent updates in 4D human perception, which includes disentangling the camera and the human motion from challenging in-the-wild videos with multiple people. Our approach takes advantage of background pixels as cues for camera motion, which when combined with motion priors and inferred ground planes can resolve scene scale and depth ambiguities up to an “anthropometric” scale. I will also talk about nerf.studio, a modular open-source framework for easily creating photorealistic 3D scenes and accelerating NeRF development. I will discuss our recent works, which highlight how language can be incorporated for editing and interacting with the recovered 3D scenes.

Bio: Angjoo Kanazawa is an Assistant Professor in the Department of Electrical Engineering and Computer Science at the University of California at Berkeley. Her research is at the intersection of Computer Vision, Computer Graphics, and Machine Learning, focusing on the visual perception of the dynamic 3D world behind everyday photographs and video. Previously, she was a research scientist at Google NYC, and prior to that she was a BAIR postdoc at UC Berkeley. She completed her PhD in Computer Science at the University of Maryland, College Park, where she also spent time at the Max Planck Institute for Intelligent Systems. She has been named a Rising Star in EECS and has been honored with the Google Research Scholar Award and most recently the Sloan Fellowship 2023.

Homepage:  https://people.eecs.berkeley.edu/~kanazawa/

Sponsored in part by:   Meta Reality Labs Pittsburgh