Skeleton Based Action Quality Assessment of Figure Skating Videos

Abstract

Action quality assessment(AQA) aims at achieving automatic evaluation the performance of human actions in video. Compared with action recognition problem, AQA focuses more on subtle differences both in spatial and temporal dimensions during the whole executing process of actions. However, most existing AQA methods tried to extract features directly from RGB videos through a 3D ConvNets, which makes the features mixed with useless scene information. To overcome this problem, We propose a deep pose feature learning AQA method that captured detailed and meaningful representations for skeleton information to discover the subtle motion difference of AQA problem. We first apply pose estimation method to obtain human skeleton data from RGB videos. Then a spatio-temporal graph convolutional network (ST-GCN) is employed to extract the dynamic changes of skeleton data and obtain the representative pose features. Finally, a regressor composed of three fully connected layers is developed to reduce the dimension of the obtained pose features and predict the final score. Experiments on MIT figure skating dataset have been extensively conducted, and the results demonstrate that the proposed method has achieved improvements that outperformed current state-of-the-art methods.