Human Activity Recognition in Thermal Infrared Imaging Based on Deep Recurrent Neural Networks

Human activity recognition (HAR) is a vast branch of research that focuses on determining the specific action of a person according to sensor data. However, predicting human activity at night is still challenging due to the lack of sufficient accuracy of sensor data. A new model for multimodal thermal infrared data-based HAR (MTIR-HAR) is presented in this paper which can enhance the HAR accuracy by automatically learning the human features from the raw data. Six extra deep layers are added to the recurrent neural network (RNN) to improve the performance of the HAR system at night. These layers extract the most complex features from thermal infrared imaging for classification. The sequence classification technique is applied to separately merged data. The experimental results showed that the proposed method outperformed (up to 98.0%) on the MHAD dataset than the SVM and LSTM methods. Furthermore, the method has achieved the highest accuracy rates (up to 80.2%) compared with other related results in the same DHU Night Dataset under different walking conditions.