Micro-expression Recognition Based on Attention-enhanced LSTM Neural Networks

Abstract

Micro-expression recognition is a difficult task in computer vision. Most existing micro-expression recognition methods extract facial features globally, leading to the inclusion of many irrelevant features and affecting the recognition accuracy in a negative way. In this paper, Long Short-Term Memory (LSTM) neural networks with spatial and temporal attention mechanisms are designed and employed to extract features selectively from the input sequences. Key frames are identified from the original micro-expression sequences at first. Then the VGG-Face model is used to extract the spatial features of those key frames. The spatial features of the micro-expression sequences are then fed into attention-enhanced long short-term memory neural networks, using a softmax function for the final classification. Our experiments with CASME II show that the attention-enhanced LSTM models improve the accuracy of micro-expression recognition significantly, compared to the results of several other leading methods.