Histogram Based Shape and Textural Characteristics for Facial Emotion Recognition

Emotion recognition has many applications in relation between human and machine. A facial emotion recognition framework for 6 basic emotions of happiness, sadness, disgust, surprise, anger and fear is proposed in this paper. The proposed framework utilizes the histogram estimate of shape and textural characteristics of face image. Instead of direct processing on the original gray levels of face image which may have not significant information about facial expression, the processing is done on transformed images containing informative features. The shape features are extracted by morphological operators by reconstruction and the texture ones are acquired by computing the gray-level co-occurrence matrix (GLCM), and applying Gabor filters. The use of whole face image may provide non-informative and redundant information. So, the proposed emotion recognition method just uses the most important components of face such as eyes, nose and mouth. After textural and shape feature extraction, the histogram function is applied to the shape and texture features containing emotional states of face. The simple and powerful nearest neighbor classifier is used for classification of fused histogram features. The experiments show the good performance of the proposed framework compared to some state-of-the-art facial expression methods such as local linear embedding (LLE), Isomap, Morphmap and local directional pattern (LDP).