Video-Based Deception Detection via Capsule Network With Channel-Wise Attention and Supervised Contrastive Learning

Abstract

Deception detection is essential for protecting the public interest and maintaining social order. Its application in various fields helps to establish a safer and trustworthy social environment. This study focuses on the problem of deception detection in videos and proposes a visual deception detection method based on a capsule network (DDCapsNet). The DDCapsNet model predicts deception classification using the fusion of facial expression features and video-based heart rate feature via a channel attention mechanism. Supervised contrastive learning is further introduced to enhance the generalization ability of the DDCapsNet. The proposed model is evaluated on a self-collected dataset (physiological-assisted visual deception detection dataset, PV3D) and the public Bag-of-Lies (BOL) dataset, respectively. The results show that DDCapsNet outperforms the unimodal system and other state-of-the-art (SOTA) methods, where the ACC reaches 77.97% and the AUC reaches 78.45% on PV3D, and the ACC reaches 73.19% and the AUC reaches 72.78% on BOL dataset.