Abstract: We observe and interact with myriad of objects in our everyday lives, from cups and bottles to hammers and tennis rackets. In this talk, I will outline our group’s efforts towards understanding these objects and our everyday interactions with them in 3D. I will first focus on scaling 3D prediction for isolated objects across generic categories, and describe approaches that allow coarse prediction from single view and detailed inference given multi-view. Moving beyond isolated objects, I will present approaches aimed to understanding hand-object interactions in 3D — both in terms of reconstructing and imagining them from images. Finally, I will highlight applications of this 3D inference in robot manipulation task, and outline some interesting future directions in this space.

Bio: Shubham Tulsiani is an Assistant Professor in the CMU School of Computer Science. Prior to this, he was research scientist at Facebook AI Research (FAIR). He received a PhD. in Computer Science from UC Berkeley in 2018. He is interested in building perception systems that can infer the spatial and physical structure of the world they observe.