Abstract:

Foundation models from machine learning have enabled rapid advances in perception, planning, and natural language understanding for robots. However, current systems lack any rigorous assurances when required to generalize to novel scenarios. For example, perception systems can fail to identify or localize unfamiliar objects, and large language model (LLM)-based planners can hallucinate outputs that lead to unsafe outcomes when executed by robots. How can we rigorously quantify the uncertainty of machine learning components such that robots know when they don’t know and can act accordingly?

In this talk, I will present our group’s work on developing principled theoretical and algorithmic techniques for providing formal assurances on learning-enabled robots that act based on rich sensory inputs (e.g., vision) and natural language instructions. The key technical insight is to leverage and extend powerful methods from conformal prediction and generalization theory for rigorous uncertainty quantification in a way that complements and scales with the growing capabilities of foundation models. I will present experimental validation of our methods for providing strong statistical guarantees on LLM planners that ask for help when they are uncertain, and for vision-based navigation and manipulation.

Bio:

Anirudha Majumdar is an Associate Professor at Princeton University in the Mechanical and Aerospace Engineering department and Associated Faculty in the Computer Science department. He also holds a part-time position as a Research Scientist at Google DeepMind. Majumdar received a Ph.D. in Electrical Engineering and Computer Science from the Massachusetts Institute of Technology in 2016, and a B.S.E. in Mechanical Engineering and Mathematics from the University of Pennsylvania in 2011. Subsequently, he was a postdoctoral scholar at Stanford University from 2016 to 2017 at the Autonomous Systems Lab in the Aeronautics and Astronautics department.

Majumdar is a recipient of the Sloan Fellowship, ONR Young Investigator Program (YIP) award, NSF CAREER award, Google Faculty Research Award (twice), Amazon Research Award (twice), Young Faculty Researcher Award from the Toyota Research Institute, Best Student Paper Award at the Conference on Robot Learning (CoRL), Paper of the Year Award from the International Journal of Robotics Research (IJRR), Best Conference Paper Award at the International Conference on Robotics and Automation (ICRA), and the Excellence in Teaching Award (Princeton SEAS).