Cross-Domain Knowledge Transfer for Skeleton-based Action Recognition based on Graph Convolutional Gradient Reversal Layer

Abstract

For skeleton-based action recognition, since there usually exists many nuances between different datasets, including viewpoints, the number of available joints for a skele-ton, the type of actions, etc, it hinders to apply and leverage the knowledge of a pretrained model for one dataset to an-other except retraining a new model for the target dataset. To address this issue, we propose a cross-domain knowledge transfer module based on gradient reversal layer along with adaptive graph convolutional network to effectively transfer the knowledge from one domain to another. The adaptive graph convolution module allows the proposed method to adaptively learn the topological relation between joints and is very useful for the scenarios when the numbers of skele-ton joints for the two domains are different and the topo-logical correspondences of joints are not clearly specified. With extensive experiments from NTU-RGB+D 60 to the PKU, CITI3D, and NW datasets, the proposed approach achieves significantly better results than other state-of-the-art spatio-temporal graph convolutional network methods which are trained on the target dataset only, and this also demonstrates the effectiveness of the proposed approach.