AM40: Enhancing action recognition through matting-driven interaction analysis

Action recognition models frequently face challenges from complex video backgrounds, where actors may blend into their surroundings and complicate motion analysis. Human interactions with action-related elements vary across scenarios, with backgrounds serving as both contextual cues and sources of interference. To address these issues, we introduce video matting techniques to separate foreground subjects from the background. This enables the model to focus on the subject of interest while suppressing irrelevant regions, thereby enhancing the extraction of interactions between the subject and associated objects. To support this methodology, we present ActionMatting40 (AM40) dataset, which comprises 40 action categories annotated with alpha mattes to distinguish human actions and related objects from the background. Furthermore, we propose Matting-Driven Interaction Recognition (MIR), integrating an Action Background Decoupling (ABD) module to mitigate background interference and a Semantic-aware Feature Communication (SFC) module to selectively extract informative features for improved action recognition. Our code and dataset are publicly available at https://github.com/lwxfight/actionmatting.