Automatic Micro-Expression Recognition using LBP-SIPl and FR-CNN

Facial expressions are one of the most effective ways for the non-verbal communications, which can be expressed as the Micro-Expression (ME) in the high-stake situations. The MEs are involuntary, rapid, and subtle. Therefore, they can reveal the real human intentions. However, the MEs’ feature extraction is very challenging due to their low intensity and very short duration. Although Local Binary Pattern on Three Orthogonal Plane (LBP-TOP) feature extractor is useful for the ME analysis, it does not consider the essential information. To address this problem, in this research paper, we propose a novel feature extractor called the Local Binary Pattern from Six Intersection Planes (LBP-SIPl). This method extracts the LBP code on the six intersection planes, and then it combines them. The results show that the proposed method has superior performance in the apex frame spotting automatically, in comparison with the relevant methods on the CASME I and the CASME II databases. Then, the apex frames are the input of the Fast Region-based Convolutional Neural network (FR-CNN) to recognize the facial expressions from them. The simulation results show that, using the proposed method, the ME has been automatically recognized in 81.56% and 96.11% on the CASME I and the CASME II databases, respectively.