Event Location: NSH 1507

Abstract: The goal of this thesis is to further our understanding of nonverbal communication by developing methods and representations to quantify and predict human body motion during social interactions. We design a data collection protocol and capture system to obtain 3D body pose and facial annotations for groups of interacting people engaged in a social game. The system uses a multi-view configuration of RGB-D sensors to generate 3D reconstructions of room-sized scenes.

To analyze human motion data, we present the Kronecker Markov Random Field model for point cloud representations of the face and body. We show that most of the covariance in natural body motions correspond to a specific set of spatiotemporal dependencies which result in a Kronecker or matrix normal distribution over spatiotemporal data, and we derive associated inference procedures that do not require training sequences. This statistical model can be used to infer complete sequences from partial observations, and unifies linear shape and trajectory models of prior art into a probabilistic shape-trajectory distribution that has the individual models as its marginals.

We propose to forecast human motion in social situations by modeling the social interaction as a communication system in which we posit that attention can act as a switch, controlling the flow of information between people. We will explicitly incorporate attention into a switching dynamical system describing the 3D motions of the group, and use the models’ ability to predict or forecast motion in novel situations as a measure of performance. Our overarching goal is to identify a set of body motions that have a repeatable and predictable effect on the motion of other group members. We call these motions sociemes units of motion with a social meaning.

Committee:Yaser Sheikh, Co-chair

Iain Matthews, Co-chair

Fernando De la Torre

Brooke Feeney

David Fleet, University of Toronto