Deception detection with multi-scale feature and multi-head attention in videos

Abstract

Detecting deception in videos has been a challenging task, especially in real world situations. In this study, we extracted the facial action units from the micro-expression, and then calculated the frequency and the number of occurrences of each action unit. To get more information on different scales, we proposed a combination scheme of Multi-Scale Feature (MSF) model and Multi-Head Attention (MHA). The MSF model consists of two CNN with different convolution kernels and GELU is used as the active function. The MHA model was designed to divide the input features into different subspaces and generate attention for each subspace to make the features more effective. We evaluated our proposed method on the Real-life Trial dataset and achieved an accuracy of 87.81%. The results show that the MSF and MHA model could increase the accuracy of deception detection task. And the comparative experiment demonstrates the effectiveness of our proposed method.