Enhancing Deception Detection with Exclusive Visual Features using Deep Learning

Abstract

A combination of nonverbal cues, verbal cues, and measurements of body abnormality make guidelines to determine deceitfulness. The combination of these guidelines will vary from person to person, making deception detection a complex challenge. Research has demonstrated that the accuracy of the latest computerized polygraph testing techniques is 98% accurate. Several human-controlled variables help to achieve this level of accuracy, such as being properly trained and must use an accepted procedure and scoring system from the British Polygraph Society. This causes a lack of availability for Deception detection as the implementing these techniques have training from the British Polygraph Society. Hence this research aims to reduce the requirements of lie detection by relying on Visual Features tracked with computer vision. The proposed multi-modal will track facial and body movements to classify whether a person is Deceiving or telling the Truth. The model proposed will use data consisting of videos collected from public court trials. The data will be cleaned with Facial Action Units (AU) with OpenFace, and then augmented with various rotations. The features extracted from the videos are the Movement with Holistic landmarks and Unique features from deep learning extraction. The Multi-model will consist of three pathways: a 3D-CNN pathway, a CovLSTM2D Pathway, and a dense pathway. The outputs of the three paths are concatenated and fed into a dense layer with SoftMax activation for classification. With a continuous emphasis on examining the proposed methodology for model creation, we discovered that higher accuracy can be achieved by leveraging deep learning algorithms for visual inputs as complex as the human body.