Predicting and Classifying Pedestrian Behavior Using an Integrated Cognitive Architecture

Abstract

We present an integrated system that combines a state-of-the-art object detection algorithm with the ACT-R cognitive architecture. This system represents the first step towards a more complete integration of perception and cognition for solving real-world tasks. We use this system to detect pedestrians on a sidewalk and classify and predict their behavior. Perception provides a bounding box for each pedestrian detected at each time-step. ACT-R uses this information to track the pedestrians across the scene. Simultaneously, ACT-R checks the tracks against a set of spatial features. During the learning phase, the model learns to associate sequences of these features with the appropriate behavior. During classification, the detected feature sequence is used to retrieve the appropriate behavior from memory. During prediction, partial detected feature sequences are used to retrieve the associated behavior from memory. We provide results of classification and prediction for single and multiple behavior sets, and discuss future work.