A Review of Micro-expression Recognition based on Deep Learning

Abstract

Micro-expression has the characteristics of spontaneity, low intensity, and short duration, which reflects a real personal emotion. Therefore, micro-expression recognition (MER) has been applied widely in lie detection, depression analysis, human-computer interaction systems, and commercial negotiation. Micro-expressions usually occur when people attempt to cover up their true feelings, especially in high-stake environments. In the early stage, the study of micro-expressions was mainly from a psychological point of view and required a very specialized skill. MER based on deep learning is a hot research direction recently, which generally includes several stages, such as image preprocessing, feature extraction, and emotion classification. In this paper, we first introduce the problems and challenges MER encountered. Then we present the commonly used micro-expression datasets and methods of image preprocessing. Next, we describe the MER methods based on deep learning in recent years and classify them according to the network structure. Afterward, we present the evaluation metrics and protocol and compare different algorithms on the composite dataset. Finally, we conclude and provide a prospect of the future work of MER.