Violence detection in video game metadata using ConvLSTM

The automatic detection of violent situations is relevant to monitor exposure to violence, both in the context of the analysis of real video and video generated in virtual environments, namely in simulated scenarios or virtual/mixed reality applications, such as serious games. In this paper, we propose a deep neural network to identify violent videos, with an approach capable of working in real video and synthetic video. An efficient detector of the 2D pose together with a multiple person tracker is used to extract motion features from the video sequence that will be fed directly into the proposed network. The proposed convolutional neural network is a recurrent neural network for a Spatio-temporal prediction that has convolutional structures in both the input-to-state and state-to-state transitions, which enables the analysis of local motion in the video. By stacking multiple ConvLSTM layers and forming an encoding-forecasting structure, we obtain a network model for the violence detection problem and more general spatiotemporal sequence forecasting problems. The inputs for the model correspond to sequences of keypoints extracted from the skeletons present in each frame originating an output corresponding to the classification of the video. The model was trained and evaluated with an innovative dataset that contains violent videos from a popular fighting game and non-violent videos related to people's daily lives. Comparison of the results obtained with the state-of-the-art techniques revealed the promising capability of the proposed method in recognizing violent videos with 100% precision, although it is not as robust as other datasets. Conv-LSTM units are shown to be an effective means for modelling and predicting video sequences.