A Multidimensional Feature Extraction and Fusion Framework Based on Aggregation and Temporal Adaptation for Human Activity Recognition

Abstract

Recent years have witnessed the conspicuous prosperity of deep neural networks in sensor-based human activity recognition (HAR). Nonetheless, some existing HAR frameworks based on deep learning (DL) architectures still face challenges in effectively extracting valid features and adaptively capturing complex dynamic information. Accordingly, most of the methods struggle to classify confusable activities. To settle the above challenges, a novel HAR framework for multidimensional feature extraction and fusion based on aggregation and temporal adaptation (MFEF-ATA) is proposed in this article. To construct the framework, initially, an aggregation transformation-based dual path module (ATDPM) is developed. Besides, a residual temporal bidirectional module (ResTBM) is presented, which is the residual connection of the temporal adaptive module (TAM) and bidirectional gated recurrent unit (Bi-GRU). Meanwhile, we construct a smart home activity (SHA) dataset to enrich the HAR sensor datasets from different application scenarios. The evaluation experiments of the MFEF-ATA framework are carried out on the wireless sensor data mining (WISDM) dataset, the University of California, Irvine HAR (UCI-HAR) dataset, and the SHA dataset. The experimental results show that the MFEF-ATA framework can derive better recognition performance than other state-of-the-art HAR frameworks with recognition accuracies of 99.12%, 97.77%, and 98.52% on the WISDM dataset, the UCI-HAR dataset, and the SHA dataset, respectively, which proves the effectiveness and superiority of the proposed framework.