Fishy Action Recognition from Cross-Event Integrated Video Dataset Based on Deep 3D and Mixed CNN models

Abstract

As two crucial aspects- 1) temporal features and 2) motion characteristics are to be scrutinized apart from spatial content in analyzing video actions, therefore applying the two-dimensional CNNs alone entail fundamental problems: They do not consider the temporal ordering and eventually collapse the temporal correlations. In this paper, an automatic deep fishy action recognition system has been developed. We have implemented two spatiotemporal convolutional algorithms (pre-trained on Kinetics-400) that are capable of handling information regarding temporal, spatial, and motion in video sequences via residual learning techniques. Six categories of suspicious videos where there are only fine-grained atomic actions from five different datasets have been considered and a composite dataset has been created. To reduce over-fitting and making the dataset more robust and generalized, synthetic data have been procured through the offline augmentation processes. To the best of our knowledge this is the first attempt to recognize malicious actions across an integrated dataset comprising of five cross-event and heterogeneous datasets of actions. Therefore, we had to combat with multifarious challenges of domain shifting. We have achieved the higher level of accuracy.