Abstract: In constraint tasks, the current task state heavily limits what actions are available to an agent. Mechanical constraints exist in many common tasks such as construction, disassembly, and rearrangement and task space constraints exist in an even broader range of tasks. Deep reinforcement learning algorithms have typically struggled with constraint tasks for two main reasons: the sequential nature of these tasks makes encountering the final reward difficult and transferring information between task variants using continuously learned parameters rather than discrete symbols is inefficient. In this work, we present a memory-based learning solution that leverages the discrete, symbolic nature of task constraints, to quickly acquire high-level information about a task and transfer it to task variants. We show that our method generalizes to unseen task variants an order of magnitude faster than deep reinforcement learning methods.

As a side note, birds are very good at solving constraint tasks and generalizing their knowledge. To see some cool videos of birds solving mechanical locking puzzles, and to see some of our inspiration, check out this video and this video.

Committee:
Chris Atkeson
Oliver Kroemer
Dave Held
Leo Keselman