Seminar
Is Data All You Need?: Large Robot Action Models and Good Old Fashioned Engineering
Abstract: Enthusiasm has been skyrocketing for humanoids based on recent advances in "end-to-end" large robot action models. Initial results are promising, and several collaborative efforts are underway to collect the needed demonstration data. But is data really all you need? Although end-to-end Large Vision, Language, Action (VLA) Models have potential to generalize and reliably solve [...]
The New Era of Video Generation
Abstract: Traditional video production is slow, expensive, and requires specialized skills. Founded by CMU alumni, HeyGen is an AI-native video platform designed to revolutionize the video creation process by making visual storytelling accessible to all. We've successfully grown to more than 20M users, and tens of millions revenue in less than one year, with recognition [...]
Sensing the Unseen: Dexterous Tool Manipulation Through Touch and Vision
Abstract: Dexterous tool manipulation is a dance between tool motion, deformation, and force transmission choreographed by the robot's end-effector. Take for example the use of a spatula. How should the robot reason jointly over the tool’s geometry and forces imparted to the environment through vision and touch? In this talk, I will present our recent [...]
Autoregressive Models: Foundations and Open Questions
Abstract: The success of Autoregressive (AR) models in language today is so tremendous that their scope has, in turn, been largely narrowed to specific instantiations. In this talk, we will revisit the foundations of classical AR models, discussing essential concepts that may have been overlooked in modern practice. We will then introduce our recent research [...]
Learning Environment Models for Mobile Robot Autonomy
Abstract: Robots are expected to execute increasingly complex tasks in increasingly complex and a priori unknown environments. A key prerequisite is the ability to understand the geometry and semantics of the environment in real time from sensor observations. This talk will present techniques for learning metric-semantic environment models from RGB and depth observations. Specific examples include [...]
Physical Intelligence and Cognitive Biases Toward AI
Abstract: When will robots be able to clean my house, dishes, and take care of laundry? While we source labor primarily from automated machines in factories, the penetration of physical robots in our daily lives has been slow. What are the challenges in realizing these intelligent machines capable of human level skill? Isn’t AI advanced [...]
