Skeleton-Based Motion Recognition for Labanotation Generation Based on the Fusion of Neural Networks

Abstract

Labanotation is a scientific method for documenting dance movements that has been widely adopted globally. Existing methods for Labanotation action recognition perform poorly in handling complex movements and integrating spatiotemporal information. To address this, we propose a multi-branch spatiotemporal fusion network with attention mechanisms aimed at accurately recognizing Labanotation actions from motion capture data. Initially, we convert motion capture data into three-dimensional coordinates and extract skeleton vector features. Subsequently, we enhance feature representation by extracting temporal difference features and skeleton angle features from the skeleton vectors. These features are processed using gated recurrent units and residual networks to effectively integrate spatiotemporal information. Finally, attention mechanisms are applied in the model to differentiate the importance of different positions in the features. This method effectively models spatiotemporal relationships, thereby improving the accuracy of Labanotation action recognition. We conducted experiments on two segmented motion capture datasets, demonstrating the effectiveness of each module. Compared to existing methods, our approach shows superior performance and strong generalization ability. Given the relative simplicity of upper limb action recognition, our focus primarily lies on lower limb action recognition. Notably, this marks the first application of skeleton angle features in the field of Labanotation action recognition.