Through-Wall Human Activity Recognition With Dual-Layer Attention Augmented Multiscale Multiview Feature Fusion Network Using Low-Frequency Multistatic Bio-Radar

Abstract

The anisotropy of both multiview detection paths and human targets presents a challenge for the through-wall detection and classification of enclosed space-high degrees of freedom (ES-HDR) human behavior, characterized by random positions and orientations. The obtained multiview micro-Doppler signatures (MDs) spectra may exhibit varying levels of information quality at both the feature spectrum space layer and the view space layer. If all the information contained at either layer is treated equally, this will seriously limit the fusion recognition results, that is, the phenomenon of “multiview fusion heterogeneous reverse autophagy.” In this article, a dual-layer attention (DA) augmented multiscale multiview feature fusion network is proposed for through-wall human activity recognition based on multiview MDs spectra. Specifically, during the single-view MDs spectra feature extraction stage, a multiscale attention feature fusion module (MSAM) is utilized to integrate features from different depth levels, where SpectralSpace Attention (SSA) is utilized to focus on the information in the feature spectrum space layer at the corresponding depth levels and dynamically assigns weights based on the importance differences. During the multiview feature fusion stage, a multiview attention feature fusion module (MVAM) is utilized to fuse multiview information, where ViewSpace Attention (VSA) is utilized to adaptively focus on the important views. The DA mechanism works together to mitigate the phenomenon of “multiview fusion heterogeneous reverse autophagy,” effectively improving the recognition performance of multiview through-wall detection ES-HDR human behavior. Experimental results indicate that the proposed method achieves 98.542% recognition accuracy for seven types of through-wall human activities. Ablation experiments and visualization analysis strongly demonstrate the effectiveness of DA and its ability to improve the fusion recognition performance.