Violence detection in crowd videos using nuanced facial expression analysis

Abstract

Video analysis for violence detection is crucial, especially when dealing with crowd data, where the potential for severe mob attacks in sensitive areas is high. This paper proposes a solution utilizing Convolutional Restricted Boltzmann Machine (CRBM) for video analysis, integrating the strengths of Convolutional Neural Network (CNN) and Restricted Boltzmann Machine (RBM). By focusing on image patches rather than entire frames, the method addresses the challenge of object detection in crowded scenes. The CRBM combines deep-level image analysis from CNN with unsupervised feature extraction in RBM, facilitated by image convolution using Gabor filters in the hidden layer. Dropout regularization mitigates overfitting, enhancing model generality. Extracted features are inputted into an SVM classifier for face detection and a custom VGG16 model for emotion identification. Event probability is then determined through logistic regression based on facial expressions. Despite existing approaches for smart crowd behaviour identification, there remains a tradeoff between accuracy and processing time. Our proposed solution addresses this by employing proper frame preprocessing techniques for feature extraction. Validation using quantitative and qualitative metrics confirms the effectiveness of the approach.