Parallel Spatiotemporal Network to recognize micro-expression

Abstract

Micro-expressions are fleeting spontaneous facial expressions that commonly occur in high-stakes scenarios and reflect humans’ mental states. Thus, it is one of the crucial clues for lie detection. Furthermore, due to the brief duration of micro-expression, temporal information is important for micro-expression recognition. The paper proposes a Parallel Spatiotemporal Network (PSN) to recognize micro-expression. The proposed PSN includes a spatial sub-network and a temporal sub-network. The spatial sub-network is a shallow network with subtle motion information as the input. And the temporal sub-network is a network with a novel temporal feature extraction unit that extracts sparse temporal features of micro-expressions. Finally, we propose an element-wise addition with 1 × 1 convolutional kernel fusion model to fuse the spatial and temporal features. The proposed PSN gets better measurement metrics (such as recognition rate, F1 score, true positive rate, and true negative rate) than the other state-of-the-art methods on the consisted databases consisting of CASME, CASME II, CAS(ME)${}^2$, and SAMM.