KMANet: A spatio-temporal enhancement network for micro-action recognition

Abstract

Action recognition technology has gained widespread application due to its ability to capture and process fine-grained motion details. Recent research has increasingly focused on analyzing individual emotions and intentions, bringing greater attention to micro-action recognition (MAR), which involves subtle and low-intensity movements. However, MAR faces several challenges, such as subtle variations in motion amplitude and highly similar visual features. These factors limit the effectiveness of traditional action recognition methods in achieving high detection accuracy. To address these limitations, we drew inspiration from the MAR benchmark MANet and focused on temporal feature modeling and effectively discriminative regions of micro-actions. Accordingly, we propose a two-stage MAR framework with a collaborative mechanism, termed KMANet, which adopts a two-stage spatiotemporal feature enhancement strategy. Specifically, in the temporal dimension, we design a Key Frame Attention Mechanism (KFAM) to automatically focus on key-frame sequences of micro-actions and capture inter-frame dynamic relationships, thereby reducing the interference of non-essential frames. This approach effectively addresses the issue of insignificant motion amplitude changes. The integration of the Micro-Action Focus Module (MAFM) on top of KFAM serves to further enhance local spatial features and reinforce detailed representation in core motion regions. The integration of these two modules achieves a substantial improvement in recognition accuracy at a minor computational expense. Extensive experimentation on the MAR dataset MA-52 and BBSI demonstrates that, in comparison to state-of-the-art methods, KMANet fulfills the requirements of fine-grained scenario detection and attains superior recognition accuracy and performance in micro-action recognition tasks.