Event Location: NSH 3002

Abstract: To make intelligent decisions, robots often use models of the effects of their actions on the world. Unfortunately, in complex environments, it is often infeasible to create models that are accurate in every plausible situation, which can lead to suboptimal performance. In these cases, online correction of model inaccuracies can lead to substantial performance improvement. Furthermore, because of high domain dimensionality, expensive testing, and finite-horizon requirements, robots need to make these corrections from sparse execution observations. We propose to explore the problem of model correction from sparse observations for robots with situation-dependent model inaccuracies. In particular, we propose to model these inaccuracies as Regions of Inaccurate Modeling (RIMs) in the robot’s state-action space.

First, we enable our robots to detect and correct RIMs online by monitoring their execution of robots, comparing model-generated expectations to execution-generated observations. Our algorithms detect RIMs as by explicitly searching for parametric regions of state-action space in which observations deviate statistically-significantly from the nominal model. We propose to extend our algorithms to detect RIMs in high-dimensional spaces. We will assume that RIMs are intrinsically low-dimensional, but often embedded in high-dimensional spaces. This will allow us to adapt techniques from dimensionality reduction to extract, at execution time, the dimensions of space that are relevant to RIM-detection.

Then, we propose to address the problem of planning for model refinement in domains with RIMs, with the goal of finite-horizon task-performance improvement. To do this effectively, first we will enable robots to represent uncertainty about the extent of RIMs in their domain. Then, we will adapt methods from Reinforcement Learning to account for RIMs in the robot’s domain. This will enable our robots to appropriately balance actions to refine their models with actions to get immediate reward. We will evaluate the resulting algorithms in the adversarial domain of autonomous robot soccer. In this domain, our robots’ models of the opponents are inevitably inaccurate in some situations; furthermore, our robots only have a 20-minute game, and thus very sparse observations, to detect RIMs, avoid playing into opponents’ detected strengths, and exploit their weaknesses.

Committee:Reid Simmons, Co-chair

Manuela Veloso, Co-chair

Jeff Schneider

Brian Williams, Massachusetts Institute of Technology