Abstract:
Intelligent vehicles have been proposed as one path to increasing vehicular safety and reduce on-road crashes. Driving intelligence has taken many forms, ranging from simple blind spot occupancy or forward collision warnings to lane keeping and all the way to full driving autonomy in certain situations. Primarily, these methods are outward-facing and operate on information about the state of the vehicle and surrounding traffic elements. However, another less explored domain of intelligence is cabin-facing information such as driver’s state.

In this thesis, we investigate the utility of one such signal: driver eye gaze. Eye gaze allows us to infer driver internal states and we explore how this can improve both autonomous driving and intelligent driving assistance. We first present DReyeVR, an open-source virtual reality driving simulator, which was designed with behavioural and interaction research priorities in mind. Next, we use DReyeVR to conduct a psychophysical experiment which helps us characterize the extent and dynamics of driver peripheral vision. Subsequently, we use driver gaze to provide additional supervision to autonomous driving agents trained via imitation learning in an effort to mitigate causal confusion. Finally, we propose two user studies to help us build and evaluate a model of driver situational awareness (SA). The first will be a data collection effort, using a novel SA labeling method, to obtain continuous, per-object driver awareness labels. We will use this data to train a model to predict drivers’ situational awareness of traffic elements given a history of their gaze and the scene context. The second user study will be an online evaluation of the effectiveness of assistive alerts provided to drivers when their SA is deficient according to our model.

Thesis Committee Members:
Henny Admoni, Chair
David Held
Nikolas Martelaro
Chien-Ming Huang, Johns Hopkins University

More Information