Abstract:

Generative modelling has been extremely successful in synthesizing text, images, and videos. Can the same machinery also help us better understand how to physically interact with the multimodal 3D world? In this talk, I will introduce some of my group’s work in answering this question. I will first discuss how we can enable 2D image generation models to edit images in a 3D-aware manner, and how to generate audio for muted egocentric videos. I will then zoom in specifically on hand interactions by introducing (1) FoundHand, a large-scale generative model for synthesizing realistic 2D hand images, and (2) GigaHands, a new large-scale 3D hand activities dataset designed to push the boundary of hand interaction modeling. Finally, I will conclude with an outlook of the future of generative modeling for understanding 3D human interactions.

Bio:

Srinath Sridhar (https://srinathsridhar.com) is an Assistant Professor of Computer Science at Brown University, where he leads the Interactive 3D Vision & Learning Lab (https://ivl.cs.brown.edu). He received his PhD at the Max Planck Institute for Informatics and was subsequently a postdoctoral researcher at Stanford. His research interests are in 3D computer vision and machine learning. Specifically, his group focuses on visual understanding of 3D human physical interactions with applications ranging from robotics to mixed reality. He is a recipient of the NSF CAREER award, a Google Research Scholar award, and his work received the Eurographics Best Paper Honorable Mention. He spends part of his time as a visiting academic at Amazon Robotics, and has previously spent time at Microsoft Research Redmond and Honda Research Institute.

Homepage:  https://srinathsridhar.com/