Human Behaviour Recognition Method Based on SME-Net

Abstract

Spatiotemporal, motion and channel information are pivotal in video-based behaviour recognition. Traditional 2D CNNs demonstrate low computational complexity but fail to capture temporal dynamics effectively. Conversely, 3D CNNs excel in recognising temporal patterns but at the cost of significantly higher computational demands. To address these challenges, we propose a generic and effective SME module composed of three parallel sub-modules, namely Spatio-Temporal Excitation (STE), Motion Excitation (ME) and Efficient Channel Excitation (ECE). Specifically, the STE module enhances the spatiotemporal representation using a single-channel 3D convolution, enabling the model to focus on both temporal and spatial features. The ME module emphasises motion-sensitive channels by calculating feature map differences at adjacent time steps, guiding the model toward motion-centric regions. The ECE module efficiently captures cross-channel interactions without dimensionality reduction, ensuring robust performance while significantly reducing model complexity. Pre-trained on the ImageNet dataset, the proposed method achieved Top-1 accuracy of 49.0% on the Something-Something V1 (Sth-Sth V1) dataset and 40.8% on the Diving48 dataset. Extensive ablation studies and comparative experiments further demonstrate that the proposed method strikes an optimal balance between recognition accuracy and computational efficiency.