Simultaneous registration and clustering for temporal segmentation of facial gestures from video

Abstract

Temporal segmentation of facial gestures from video sequences is an important unsolved problem for automatic facial image analysis. At least two problems contribute to the challenge of temporal segmentation. These are the difficulty to register the rigid and non-rigid motion of the face, and the large variability in temporal scale of facial gestures. To address these challenges, we propose a two-step approach to temporally segment facial gestures. The first step clusters shape and appearance features invariantly to geometric transformations using Parameterized Cluster Analysis (PaCA). PaCA is a novel method that jointly performs registration and clustering. The second step temporally groups the resulting clusters into temporally coherent facial gestures. Analysis of simulated and real examples illustrates the benefits of our approach for temporal facial gesture segmentation.