Dynamic facial expression understanding based on temporal modeling of transferable belief model

Abstract

In this paper we present a novel approach for dynamic facial expressions classification. This work is in the continuity of our previous work on static facial expression classification based on the Transferable Belief Model. The system is able to recognize pure as well as mixture of facial expressions (Joy, Surprise, Disgust and Neutral) and to deal with all facial feature configurations which does not correspond to any of the cited expression (Unknown expressions). Here we present a major improvement of this former work consisting in the introduction of the temporal evolution of the facial feature behavior during a facial expression sequence. The temporal information is introduced first to improve the robustness of the frame-by-frame classification by the correction of errors due to the automatic segmentation process. Secondly, a facial expression is the result of a dynamic and progressive combination of facial features behavior which is not always synchronous. Then a frame-by-frame classification is not sufficient. Here the introduction of the temporal information inside the TBM fusion framework allows to tackle this problem. The recognition is accomplished by combining all facial feature behaviors between the beginning and the end of an expression sequence independently to their chronological order then the final decision is taken on the whole sequence. Consequently the recognition becomes more robust and accurate. Experimental results on the Hammal Caplier database demonstrate the improvement on the frame-by-frame facial expressions classification and the ability to recognize entire facial expression sequences. Finally the system is able to automatically display rich and detailed informations on the facial feature behaviors during an expression sequence.