Causal Robot Learning for Manipulation

Abstract

Two decades into the third age of artificial intelligence, the rise of deep learning has yielded two seemingly disparate realities. In one, massive accomplishments have been achieved in deep reinforcement learning, protein folding, and large language models. Yet, in the other, the promises of deep learning to empower robots that operate robustly in real-world environments have yet remained unfulfilled. Vast diversity of objects, distribution shifts, long-tailed phenomena: outside of laboratories, real-world environments challenge modern statistical learning assumptions of the data.

Although such environments have generally been referred to as "unstructured," this terminology belies their nature. Real-world environments are not "unstructured," but arise because of structure: the underlying causal processes that generate the observed data. In this view, robots should not only reason and learn with respect to data, but also the data generating processes. Such processes can be formalized by the language of causality. Therefore, to learn and leverage the structure of these "open-world" environments, new causal-based robot learning algorithms are needed.

Towards this goal, this thesis explores a diversity of robot learning problems, from perception to control. First, we explore how perception models can be learned using sim-to-real transfer from synthetic data (DREAM, FormNet). In these works, relevant features were learned through domain randomization, leading to insights into how structure can be learned more directly using causality. To this end, we introduce structural sim-to-real transfer, where simulation can serve as a causal reasoning engine for the robot to select the relevant features for a control policy (CREST) or skill (SCALE). By construction, these policies are robust to irrelevant distribution shifts that would otherwise stymie correlation-based deep learning. Next, the rich interplay between control, dynamical systems, and causality is explored through the Learning By Doing (LBD) competition and LMeshNet, a methodology for constructing hybrid causal world models that integrate both latent and semantic information. Lastly, we examine curriculum learning. In ACL, we explore the commonalities and differences between human and agent curriculum learning. Then, we employ these lessons learned for CURATE: how agents can manipulate the sequencing of training data to efficiently learn a control policy to solve a desired, difficult task.

The advantages of adopting the principles of causal inference have been witnessed to date in fields such as biomedical sciences, economics, and genomics. In the machine learning community, it has recently been argued that such principles should be integrated to harness deep learning, towards causal learning of representations. Analogously, this thesis forwards that the robot learning community stands to gain by leveraging the principles of causality. In so doing, this new paradigm holds promise for robots to learn and leverage structure within the open world through causal robot learning for manipulation.