Adaptive Temporal Segmentation for Action Recognition

Abstract

Learning deep representations have been widely used in action recognition task. However, the features extracted by deep convolutional neural networks (CNNs) have many redundant information. This paper aims to discover the relevance between temporal features extracted by CNNs. Different fromTemporal Segment Networks (TSN) to randomly select video clips. Based on the matrix-based Rényi’s α-entropy, we estimate the mutual information between temporal domain features. Through our experiments, we propose an adaptive temporal segmentation scheme to represent the entire videos. We also combine the features of RGB and optical flow frames extracted by 3D ConvNets to verify the complementary information between them. We show that the proposed approach achieves 94.4 and 72.8 percent accuracy, in the UCF- 101 and HMDB-51 datasets.