Label-efficient head-motion-robust wearable radar hand gesture recognition via motion-domain radar–IMU fusion

Abstract

Hand gesture recognition (HGR) is an important input modality for augmented reality, assistive interfaces, industrial hands-free control, and human–computer interaction. Although wearable image sensors provide rich appearance information, they suffer from privacy concerns, illumination changes, limited field of view, hand occlusion, and high computational burden for always-on operation. Wearable radar is an attractive non-image alternative because it directly measures range, angle, and Doppler motion cues. However, when radar is mounted on a wearable platform, head motion changes the sensor coordinate system and degrades recognition robustness. This paper studies radar–IMU fusion and label-efficient training for head-motion-robust wearable radar HGR. We compare conventional fusion strategies, including radar-only learning, early fusion, late fusion, and attention fusion, with the proposed motion-domain protocol that converts radar observations into a Doppler-aware motion-XYZ representation. Under the same TCN backbone and subject-disjoint protocol, the proposed motion-domain protocol achieved the best four-class performance among the evaluated fusion strategies, with 85.97% macro-F1 and 81.69% moving-head macro-F1. It improved moving-head macro-F1 by 3.18 percentage points over early fusion and by 5.80 percentage points over attention fusion. Label-efficiency experiments showed a larger benefit under limited non-static head labels: with 0% non-static labels, early fusion achieved only 23.46% moving-head macro-F1, whereas the proposed protocol achieved 70.30%. These results indicate that raw IMU concatenation or attention-based sensor fusion is less effective than transforming radar observations into a physically meaningful motion domain before learning.