Skeleton-Aware Radar-Based Gait Recognition Network With Cross-Modal Supervision

Owing to the significant advantages of radar sensing, gait recognition based on radar point clouds (RPCs) has gained attention as a promising noninvasive human identification solution. However, it is difficult for RPCs to provide sufficient characterization of human gait features for the specular reflection and inconsistency across multiple frames. To address this issue, a skeleton-aware radar-based gait recognition network with cross-modal supervision is proposed, which is named SRGaitNet. To improve RPC-based gait characterization, two key points are considered in the proposed network. On the one hand, the understanding of human structure from the visual modality is leveraged to guide the extraction of human pose-aware features from RPC sequences. On the other hand, to effectively make use of the extracted human skeleton and enhance gait features, a transformer-based feature fusion module is designed to aggregate the spatial-temporal and human pose features. The SRGaitNet is comprised of three modules: the first is a base module for spatial-temporal feature extraction from RPC sequences, the second is a pose estimator for human skeleton reconstruction, and the last is a transformer-based fusion module for integrating skeletal and spatial-temporal features. The network is jointly trained on dual-tasks and optimized by a combination of human identification cross-entropy and pose estimation mean square error loss. Comprehensive experimental evaluation demonstrates that the SRGaitNet is superior to existing RPC-based gait recognition methods, with an average accuracy of 85.24% over different walking paths.