AM40: Enhancing action recognition through matting-driven interaction analysis

IF 7.6 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Pattern Recognition Pub Date : 2025-09-17 DOI:10.1016/j.patcog.2025.112393

Siqi Liang , Wenxuan Liu , Zhe Li , Kui Jiang , Siyuan Yang , Chia-Wen Lin , Xian Zhong

{"title":"AM40: Enhancing action recognition through matting-driven interaction analysis","authors":"Siqi Liang , Wenxuan Liu , Zhe Li , Kui Jiang , Siyuan Yang , Chia-Wen Lin , Xian Zhong","doi":"10.1016/j.patcog.2025.112393","DOIUrl":null,"url":null,"abstract":"<div><div>Action recognition models frequently face challenges from complex video backgrounds, where actors may blend into their surroundings and complicate motion analysis. Human interactions with action-related elements vary across scenarios, with backgrounds serving as both contextual cues and sources of interference. To address these issues, we introduce video matting techniques to separate foreground subjects from the background. This enables the model to focus on the subject of interest while suppressing irrelevant regions, thereby enhancing the extraction of interactions between the subject and associated objects. To support this methodology, we present <span>ActionMatting40</span> (<span>AM40</span>) dataset, which comprises 40 action categories annotated with alpha mattes to distinguish human actions and related objects from the background. Furthermore, we propose Matting-Driven Interaction Recognition (MIR), integrating an Action Background Decoupling (ABD) module to mitigate background interference and a Semantic-aware Feature Communication (SFC) module to selectively extract informative features for improved action recognition. Our code and dataset are publicly available at <span><span>https://github.com/lwxfight/actionmatting</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49713,"journal":{"name":"Pattern Recognition","volume":"172 ","pages":"Article 112393"},"PeriodicalIF":7.6000,"publicationDate":"2025-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Pattern Recognition","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0031320325010544","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}

引用次数: 0

Abstract

Action recognition models frequently face challenges from complex video backgrounds, where actors may blend into their surroundings and complicate motion analysis. Human interactions with action-related elements vary across scenarios, with backgrounds serving as both contextual cues and sources of interference. To address these issues, we introduce video matting techniques to separate foreground subjects from the background. This enables the model to focus on the subject of interest while suppressing irrelevant regions, thereby enhancing the extraction of interactions between the subject and associated objects. To support this methodology, we present ActionMatting40 (AM40) dataset, which comprises 40 action categories annotated with alpha mattes to distinguish human actions and related objects from the background. Furthermore, we propose Matting-Driven Interaction Recognition (MIR), integrating an Action Background Decoupling (ABD) module to mitigate background interference and a Semantic-aware Feature Communication (SFC) module to selectively extract informative features for improved action recognition. Our code and dataset are publicly available at https://github.com/lwxfight/actionmatting.

Abstract Image

查看原文本刊更多论文

AM40：通过抠图驱动的交互分析增强动作识别

动作识别模型经常面临复杂视频背景的挑战，其中演员可能会融入周围环境并使动作分析复杂化。人类与行动相关元素的互动因场景而异，背景既是情境线索，也是干扰源。为了解决这些问题，我们引入了视频抠图技术，将前景主体与背景分开。这使得模型能够专注于感兴趣的主题，同时抑制不相关的区域，从而增强主题和相关对象之间交互的提取。为了支持这种方法，我们提出了ActionMatting40 （AM40）数据集，该数据集包括40个用alpha mattes注释的动作类别，以区分人类动作和背景中的相关对象。此外，我们提出了抠图驱动交互识别（MIR），集成了一个动作背景解耦（ABD）模块来减轻背景干扰，一个语义感知特征通信（SFC）模块来选择性地提取信息特征，以改进动作识别。我们的代码和数据集可以在https://github.com/lwxfight/actionmatting上公开获取。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Pattern Recognition 工程技术-工程：电子与电气

CiteScore

14.40

自引率

16.20%

发文量

683

审稿时长

5.6 months

期刊介绍： The field of Pattern Recognition is both mature and rapidly evolving, playing a crucial role in various related fields such as computer vision, image processing, text analysis, and neural networks. It closely intersects with machine learning and is being applied in emerging areas like biometrics, bioinformatics, multimedia data analysis, and data science. The journal Pattern Recognition, established half a century ago during the early days of computer science, has since grown significantly in scope and influence.