Abstract:
The promised future filled with robots sensing and acting intelligently in the world is near fruition, thanks in part to continuous progress in robotic perception. However, a number of challenges remain before robots and their perception systems can be truly reliable. In particular, we must consider what happens when highly complex perception systems designed and validated in laboratory and test environments enter the unbounded variety of reality. Will these systems fail innocuously or catastrophically? If so, how can we avoid or eliminate these failures to achieve reliable, robust perception?

The premise of this thesis is that engineering constraints and human finitude result in perception systems that cannot compensate for all possible factors and situations. We refer to the collection of uncompensated factors as the “context” of a system, and propose that variations in context can explain why it is difficult to make perception reliable at scale. Vexingly, since context is, by nature, unknowable and unmodeled, we cannot rely on prediction and foresight to compensate for it.

Instead, this thesis proposes that perception systems can adapt their behavior after deployment to the operating site to correct for unknown contextual effects. An example of this is the widespread and common practice of “parameter tuning”, typically performed by a human expert to specialize a system to each deployment. To generalize this and other mechanisms of adaptation, we first develop a general theory of context in perception and establish a statistical definition for perception performance. We then develop a practical method for evaluating performance on-site without supervision, enabling perception systems to observe the effects of context during operation. Finally, we explore automatic parameter tuning and experience-driven failure prediction as two methods of general adaptation. We demonstrate and validate this work on state estimation systems using offline data from an instrumented automobile as well as online an indoor ground robot.

More Information

Thesis Committee Members:
George Kantor, Chair
Sebastian Scherer
Katharina Muelling
Ingmar Posner, University of Oxford