Learning Temporal Structure of Videos for Action Recognition Using Pattern Theory

Aiming at the problem that a large amount of background information in the videos cause low judgment of actions, this paper proposed a graph model based on pattern theory for human complex action recognition. Firstly, a video is divided into video units and each video unit corresponds to an atomic action. The atomic action labels of videos are initialized by k-Means. Secondly, the key generator proposal module and the interpretative operation module are proposed to select important foreground information and obtain a reasonable representation of atomic action sequences. In the inference stage, the atomic action sequences of test videos are matched with template sequences by the Dynamic Time Warping algorithm (DTW) to obtain the action categories. The experimental results show that compared with the most existing human action recognition models, our model can explain the temporal process of action occurrence and obtain a more discriminatory sequence representation, which can effectively improve the accuracy of action recognition.