Synthesized GMM Free-parts Based Face Representation for Pose Mismatch Reduction in Face Verification

Abstract

Performance of face verification systems can be adversely affected by mismatches between training and test poses, especially when only one pose is available for training. Compared to holistic/monolithic representations, we show that a ?ree-parts?representation of the face is less affected by pose changes, due to: a) some patches of a subject? face retaining similar appearance across a number of different poses, and b) those patches being able to freely move position across different poses. Furthermore, we propose that this mismatch can be reduced further by synthesizing the statistical model of a subject? ?ree-parts?representation for a set of poses for which there are no gallery observations. The synthesis is accomplished by first learning how a model for a generic frontal face transforms to represent a generic face at a particular non-frontal pose. The learned transformation is then applied to each subject? frontal model to synthesize a non-frontal model. The original and synthesized models are then concatenated in order to automatically handle multiple poses.