Research on human action recognition Algorithm in infrared video based on five-dimensional bidirectional dynamic convolution and multi-head attention

Abstract

Human action recognition is a hot research topic in computer vision. In recent years, infrared imaging technology has shown unique advantages in intelligent security and health monitoring due to its night-time environmental adaptability and privacy protection features. However, existing methods face challenges such as insufficient spatiotemporal feature representation and difficulty in distinguishing highly similar actions in complex scenes due to the inherent characteristics of infrared video, including low resolution and lack of texture information. To address these issues, This paper proposes an improved model based on Five-dimensional Bidirectional Dynamic Convolution and Multi-head Attention mechanism. Firstly, to tackle the feature extraction challenge in infrared video, a five-dimensional bidirectional dynamic convolution module is designed. This module dynamically adjusts convolution kernel parameters through five types of attention weights—spatial, temporal, channel, filter, and kernel dimensions—to enhance sensitivity to low-contrast motion features. Meanwhile, deconvolutional residual connections are introduced to preserve significant spatiotemporal regions and mitigate detail loss. Secondly, to resolve the misclassification problem of highly similar actions, an efficient multi-head separable attention module is proposed. This module reduces computational overhead by sharing query and key parameters for spatial–temporal and channel attention, employs dimensionality reduction projection strategies to compress key-value matrix dimensions, and integrates depthwise separable modules to further optimize feature interaction efficiency. The comparative experiment results show that the proposed method achieved recognition accuracies of 79.37% and 87.56% on the IITR and InfAR datasets, respectively, demonstrating its superiority. The ablation experiment results indicate that our method significantly improves model accuracy and has research value. Code is available at: https://github.com/ysls160915/C3D_FBDConv_EMHSA.