Abstract: Recent breathtaking advances in AI and robotics have brought us closer to building general-purpose robots in the real world, e.g., humanoids capable of performing a wide range of human tasks in complex environments. Two key challenges in realizing such general-purpose robots are: (1) achieving “breadth” in task/environment diversity, i.e., the generalist aspect, and (2) achieving “depth” in task execution, i.e., the agility aspect. In this talk, I will present recent works that aim to achieve both generalist-level adaptability and specialist-level agility, demonstrated across various real-world robots, including full-size humanoids, quadrupeds, aerial robots, and ground vehicles. The first part of the talk focuses on learning agile and general-purpose humanoid whole-body control using sim2real reinforcement learning. The second part will discuss the limitations of such end2end sim2real pipelines and how combining learning with control can enhance safety, efficiency, and adaptability.

More details on the presented works are available on the CMU LeCAR Lab website: https://lecar-lab.github.io/