Task Level Training Signals for Learning Controllers

Abstract

Presents a new learning controller and compares it to feedback error learning on the problem of tracking with the tip of an inverted pendulum. This problem captures the important aspects of motor skills: difficult job, unstable plant, with premium placed on accuracy, efficiency, and speed of performance. The new controller abandons a traditional feedback error learning restriction and uses plant state as part of the learning process in order to retain better generalization to new curves. More importantly, it considers a new way ("coaching") to provide a target signal for training. A coach views entire task executions to create target control signals rather than using only instantaneous error and state information at each time step. In the authors' experiments, the new controller is able to track faster trajectories than plain feedback error learning.