Fast Action Recognition Based on Local and Nonlocal Temporal Feature

In this paper, we propose a mixed time-asymmetric (MTA) CNN which uses time-asymmetric convolution to extract non-local temporal feature and uses normal convolution to extract local temporal features. With the fusion of local and non-local temporal feature, our MTA CNN can achieve better action recognition accuracy while keeping the network lightweight and fast. Specially, temporal feature fusion method is designed to replace the common global average pooling in our MTA CNN so as to obtain higher-dimensional feature vector and retain more information. Extensive experimental results demonstrate that our methods can achieve comparable results on Kinetics-400 and UCF101 among leading methods with less parameters and more faster recognition speed.