An efficient gesture recognition for HCI based on semantics-guided GCNs and adaptive regularization graphs

In the embedded system, real-time gesture recognition is crucial to human–computer interaction (HCI). Recently, Graph Convolutional Networks (GCNs) have been applied to inertial measurement unit-based (IMU-based) gesture recognition. However, the disadvantage of these GCN-based methods is that they use very deep networks to capture deep motion features, without considering computational efficiency. In this paper, we propose a shallow GCN as the basic framework to ensure the real-time performance of gesture recognition. To solve the problem of shallow networks’ difficulty capturing deep motion features, we provide hand-crafted semantic information about the positions of nodes (sensors) and frames to guide deep feature extraction. Furthermore, we propose a regularization module named Double-Mask (2MASK) to enhance the network’s generalization. Experiments show that the average inference time on raspberry pi 4b is less than 4 ms. Extensive testing on the self-constructed dataset indicates that the proposed method outperforms previous state-of-the-art (SOTA) methods on multiple metrics. The accuracy reaches 89.47% and 98.70% on two public datasets, outperforming other methods. Experiments in an HCI application show that our method meets the high-precision and low-latency requirements for autonomous taxiing of UAVs. The code for this paper has been uploaded to https://github.com/oldbowls/2MAGCN-FN.