Motion Information Image for Action Recognition

Abstract

The paper proposes an effective motion information computation method, called Motion Information Image, which can aggregate motion information of multiple frames into one frame to represent the dynamic process of compact motion in video for action recognition. Motion Information Image can adopt an average sampling strategy to represent the temporal modeling relationship of the video, which allows the clever conversion of motion information into image information. Besides, we use convolutional neural networks (CNNs) combined with Motion Information Image to build a new Motion Information Image network for end-to-end action recognition. The performance of the method is improved by 2.0%~3.0% in NTU RGB+D and NTU RGB+D 120 datasets with extensive experimental validation and exhibits significant advantages for recognizing fine-grained actions.