Abstract:
Full robot autonomy is the traditional goal of robotics research. To work in a human-inhabited world, however, robots will often need to collaborate with humans. Many scenarios require human users to teleoperate robots to perform tasks, a paradigm that appears everywhere from space exploration, to disaster recovery, to assistive robotics. This collaboration enables tasks to be performed more smoothly or safely than humans could without requiring full robot autonomy. However, robots are hard to control. To compensate, roboticists build shared control systems, in which the robot operator’s command is combined with an autonomous plan to accomplish the operator’s goal.

We propose to enhance shared control systems by observing people’s natural, nonverbal behavior and using that signal to gain additional insight into their goals and concerns about the task. In particular, how people look at a scene depends on what they are thinking about the scene. Research into eye gaze behavior shows that when manipulating objects by hand, people look at their next goal or obstacle as these elements become relevant, and and they rarely look at task-irrelevant places. By tracking and processing a user’s eye gaze behavior, shared control systems can build more complex models of the user’s intentions. Equipped with this knowledge, shared control systems can both provide more accurate assistance and new types of assistance.

In this thesis, we begin by conducting a study examining how people’s eye gaze behavior relates to the task performance while they teleoperate a robot manipulator. Next, we develop a pipeline for processing the raw eye gaze sensor signal to include task context and develop models to learn aspects of user mental state from this gaze signal. Finally, we design and evaluate two gaze-based assistance systems: goal recognition, which we compare with input-based goal recognition strategies; and dynamic concern-based collision avoidance, a new approach in shared control. This thesis establishes the usefulness of the eye gaze signal for enabling more sophisticated shared control behaviors. Moreover, it shows how monitoring people’s natural behaviors can be incorporated into human-robot collaboration for more sophisticated mental state modeling and corresponding behavior.

More Information

Thesis Committee Members:
Henny Admoni, Chair
Artur Dubrawski
Nancy Pollard
Brenna Argall, Northwestern University