Two-stream Attention-aware Network for Spontaneous Micro-expression Movement Spotting

Abstract

Micro-expression is a special facial movement, which can be used as an important basis for judging people's subjective emotions. Constrained by the physiology, micro-expression can be described temporally by four phases: neutral, onset, apex, and offset. And previous studies confirmed that using the crucial temporal sequences is better than using the whole video for micro-expression recognition. Therefore, micro-expression movement spotting is considered beneficial for micro-expression recognition. While it is a challenging task due to the short duration, low intensity and usually local motion characteristics of micro-expression. Inspired by the mechanism of the ventral and dorsal visual pathways in the cerebral visual cortex, we propose an end2end two-stream attention-aware network for micro-expression movement spotting in this paper. We construct a spatial-temporal cascaded network for each stream which combines convolutional neural network and attention-aware bilateral long short-term memory recurrent neural network. And we apply the attention mechanism to two-stream feature fusion. Experiments are conducted on three available published micro-expression datasets (SMIC2, CASME, and CASME II). The experimental results show that the proposed framework outperforms state-of-the-art methods for the task of micro-expression movement spotting.