A deep learning approach for human gait recognition from time-frequency analysis images of inertial measurement unit signal

Abstract

Biomechanical analysis using deep learning has been increasingly used in recent studies to identify human activity. Wearable sensor data from inertial measurement units (IMUs) is widely used for recognizing human activity, but has several drawbacks owing to its high volume and diversity. To overcome these issues, the time-domain and power spectral characteristics of IMU data can be extracted using digital signal processing (DSP) methods. Our research aimed to investigate time-frequency analysis (TFA) methods for classifying the spatio-temporal gait characteristics of physical walking performed by healthy subjects. In this study, open-source biomechanical sensor signal dataset was used. The DSP step was first carried out by segmenting IMU data from the four body segments of 22 healthy subjects, and then by applying Continuous Wavelet Transform (CWT) and Short Time Fourier Transform (STFT) methods. Moreover, the resultants of linear accelerometer signals were applied in a similar manner. The image datasets obtained from this step were applied to a deep convolutional neural network (CNN) model to classify human walking speed (WS) into three classes: fast, normal, and slow. The performance of the 2D-CNN model and the impact of DSP methods using IMU data were evaluated. In conclusion, the highest test classification results demonstrated that STFT-all (85.9%), CWT-all (79.3%), and CWT-resAcc (76.3%) based CNN models present a remarkably precise and easy-to-implement classification problem, with the highest test accuracy, when all IMU channels are subjected to STFT. The classification accuracies of 2D-CNN models were compared to commonly used ML models. The Deep CNN model is a useful gait evaluation tool for healthy subjects. Furthermore, it can enable the diagnosis and phase assessment of gait abnormalities and detect gait biomarkers in rehabilitative wearables.