Advancing Legged Robot Agility: from Video Imitation to GPU Acceleration

Abstract

Achieving human and animal-level agility has been a long-standing goal in robotics research. Recent advancements in numerical optimization and machine learning have pushed legged systems to greater capabilities than ever before, enabling black flips, parkour, and manipulation of heavy objects. Despite these exciting developments, this thesis identifies two key limitations of current legged robot technology and aims to improve upon existing art.

First, legged robots today require manual specifications of desired behaviors and fail to learn from their human and animal counterparts. We introduce SLoMo, a first-of-its-kind framework for transferring skilled motions from casually captured videos of humans and animals to legged robots. From a monocular RGB video, SLoMo synthesizes physically plausible trajectories for downstream offline trajectory optimization and online predictive control of quadruped or humanoid robots. We demonstrate SLoMo by transferring cat and dog motions to quadruped robot hardware and human motions to a simulated humanoid robot.

Second, current model-predictive control (MPC) for legged systems often resort to simplified models due to computational limitations in real-time settings. This is due to the high dimensionality of these robots and the reliance of existing numerical optimization algorithms on fundamentally serial, CPU-friendly linear algebra routines. We leverage advancements in GPU parallelization by developing a quadratic programming (QP) solver that uses only GPU-friendly operations. We refer to our solver as ReLU-QP, thanks to its computational similarities to inferencing a deep neural network with rectified linear unit (ReLU) activation functions. Across benchmarks on solving random QPs and high-dimensional MPC tasks in simulation, including balancing a full-order Atlas humanoid robot on one foot under control limits, ReLU-QP shows an order-of-magnitude speed improvement over state-of-the-art CPU-based QP solvers and solves MPC for modern legged robots at kilohertz rates.