Disentangled adaptive multi-dimensional dynamic graph convolutional network for skeleton-based action recognition

Abstract

Skeleton-based action recognition plays a key role in computer vision and has gained significant attention due to its broad range of applications. However, most existing methods using graph convolutional networks struggle to effectively learn rich temporal and spatial motion features of body joints. In this work, the disentangled adaptive multi-dimensional dynamic graph convolutional network model that we present consists of three modules: a disentangled adaptive graph convolutional network module, a multi-dimensional dynamic temporal convolutional network module, and an efficient multi-scale attention module. Firstly, the disentangled adaptive graph convolutional network module is able to learn crucial details and interactive relationships of body joints by updating the primitive anatomical structure of the human body and adaptively changing the structural graph topology. Then, the multi-dimensional dynamic temporal convolutional network module is proposed to improve the capability of rich trajectory feature extraction and comprehensive representation. Finally, the efficient multi-scale attention module can concentrate on spatial-temporal information across the temporal and spatial dimensions to strengthen features in critical temporal frames at significant joints. Extensive experiments are performed on three large-scale datasets, including NTU RGB+D, NTU RGB+D 120, and Kinetics-Skeleton, demonstrating that the proposed model achieves state-of-the-art performance and can extract rich trajectory and spatial information from skeleton data.