End-to-End Human Motion Recognition With Multidomain Dual Attention Transformer Fusion Network and Millimeter-Wave Radar

Abstract

Human-computer interaction technology, by improving user experience and ease of use, drives innovation and growth in consumer electronics. As a noninvasive and noncontact sensing device, millimeter-wave radar has attracted great attention in human motion recognition for human-computer interaction. However, previous motion recognition models are generally based on radar echo data, i.e., images and point cloud, and single domain radar information, resulting in the loss of raw radar data information and a limited ability to capture the complementary global features. In this paper, a novel end-to-end joint global-local dual attention transformer model for human motion recognition using mmWave radar is proposed to address the above problem. First, we introduce a learnable complex transformation module to process raw radar signals for different inputs. Then, we design two important feature extraction modules, named dual residual attention module (DRAM) and dual coupled filter module (DCFM), to accurately extract the valuable motion information of the radar signal. Furthermore, a position encoding is utilized to obtain the time information of inputs feature and a transformer module is designed to get long-range context global information. Finally, the experimental results show that our proposed method achieves an average accuracy of 99.17% on the human gesture dataset and an average accuracy of 99.72% on the arm motion dataset, which demonstrates that our model has both high recognition accuracy and strong robustness.