Abstract:
Assistive robots should take actions that support people’s goals. This is especially true as robots enter into environments where personal agency is paramount, such as a person’s home. Home environments have a wide variety of “optimal’ solutions that depend on personal preference, making it difficult for a robot to know the goal it should support when entering into a new relationship with a new person in a new environment.

Furthermore, a person’s goals can change. People change their goals in a household task, such as setting a table, based on contextual information, who is coming over for dinner, their current capabilities, or even, sometimes, on a whim. These sources of variation further hinder a robot’s ability to know how to support people in advance of an interaction.

A natural solution to this problem might be for people to directly state their goals to an assistive robot prior to interaction. These goals, however, can be burdensome for people to fully describe, often requiring the production of a full task demonstration or cumbersome natural language description. During task execution though, a person’s goal directed behaviors are easily and naturally expressed and can be used to infer their goal within a task.

In this proposal, we suggest that assistive human robot interactions should be framed as collaborative inverse reinforcement learning problems, where a robot uses people’s goal directed behaviors to infer their preferred outcomes during the execution of a collaborative task.

We first provide evidence that people change their goals during assistive interactions and that online, naturalistic behaviors can be used to infer peoples’ intent through a case study of interactions with a high-degree of freedom robot arm. We then set-up the problem of robotic assistance as a multi-agent problem where the robot should take actions that align with peoples’ goals and scope the space of possible robotic assistive actions. We follow this problem definition with a formalization of assistive robotics as collaborative inverse reinforcement learning, and introduce a algorithm TILR, that aims to provide a good initialization over unknown partners and fast adaptation to individuals through their expressed behavior, which is tested in a simulated household task, which we term surface rearrangement.

Following this, we propose new contributions to extend TILR to more realistic surface rearrangement scenarios. We first focus on technical contributions that allow TILR to model large populations of varying peoples’ preferences and operate over large state spaces, a characteristic of many real-world surface rearrangement problems. These contributions should allow TILR to generalize across novel relationships and environments. We then focus on contributions that further reduce the burden on people to express their preferences by incorporating multiple modalities of preference expression, specifically language. Finally, we tie these contributions together through a study of assistive human robot interaction in a proof of concept demonstration with real people in a real surface rearrangement problem.

Thesis Committee Members:
Henny Admoni, Co-Chair
Kris Kitani, Co-Chair
Andrea Bajcsy
Dylan Losey, Virginia Tech
Chris Paxton, Meta AI

More Information