利用视频中的多尺度特征和多头注意力进行欺骗检测

IF 3 4区 计算机科学 Q2 COMPUTER SCIENCE, INFORMATION SYSTEMS
Shusen Yuan, Guanqun Zhou, Hongbo Xing, Youjun Jiang, Yewen Cao, Mingqiang Yang
{"title":"利用视频中的多尺度特征和多头注意力进行欺骗检测","authors":"Shusen Yuan, Guanqun Zhou, Hongbo Xing, Youjun Jiang, Yewen Cao, Mingqiang Yang","doi":"10.1007/s11042-024-20124-y","DOIUrl":null,"url":null,"abstract":"<p>Detecting deception in videos has been a challenging task, especially in real world situations. In this study, we extracted the facial action units from the micro-expression, and then calculated the frequency and the number of occurrences of each action unit. To get more information on different scales, we proposed a combination scheme of Multi-Scale Feature (MSF) model and Multi-Head Attention (MHA). The MSF model consists of two CNN with different convolution kernels and GELU is used as the active function. The MHA model was designed to divide the input features into different subspaces and generate attention for each subspace to make the features more effective. We evaluated our proposed method on the Real-life Trial dataset and achieved an accuracy of 87.81%. The results show that the MSF and MHA model could increase the accuracy of deception detection task. And the comparative experiment demonstrates the effectiveness of our proposed method.</p>","PeriodicalId":18770,"journal":{"name":"Multimedia Tools and Applications","volume":null,"pages":null},"PeriodicalIF":3.0000,"publicationDate":"2024-09-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Deception detection with multi-scale feature and multi-head attention in videos\",\"authors\":\"Shusen Yuan, Guanqun Zhou, Hongbo Xing, Youjun Jiang, Yewen Cao, Mingqiang Yang\",\"doi\":\"10.1007/s11042-024-20124-y\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p>Detecting deception in videos has been a challenging task, especially in real world situations. In this study, we extracted the facial action units from the micro-expression, and then calculated the frequency and the number of occurrences of each action unit. To get more information on different scales, we proposed a combination scheme of Multi-Scale Feature (MSF) model and Multi-Head Attention (MHA). The MSF model consists of two CNN with different convolution kernels and GELU is used as the active function. The MHA model was designed to divide the input features into different subspaces and generate attention for each subspace to make the features more effective. We evaluated our proposed method on the Real-life Trial dataset and achieved an accuracy of 87.81%. The results show that the MSF and MHA model could increase the accuracy of deception detection task. And the comparative experiment demonstrates the effectiveness of our proposed method.</p>\",\"PeriodicalId\":18770,\"journal\":{\"name\":\"Multimedia Tools and Applications\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":3.0000,\"publicationDate\":\"2024-09-06\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Multimedia Tools and Applications\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://doi.org/10.1007/s11042-024-20124-y\",\"RegionNum\":4,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"COMPUTER SCIENCE, INFORMATION SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Multimedia Tools and Applications","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1007/s11042-024-20124-y","RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0

摘要

检测视频中的欺骗行为一直是一项具有挑战性的任务,尤其是在现实世界中。在本研究中,我们从微表情中提取面部动作单元,然后计算每个动作单元的频率和出现次数。为了获取更多不同尺度的信息,我们提出了多尺度特征(MSF)模型和多头注意力(MHA)的组合方案。MSF 模型由两个具有不同卷积核的 CNN 组成,并使用 GELU 作为主动函数。MHA 模型的设计目的是将输入特征分为不同的子空间,并对每个子空间产生注意力,使特征更加有效。我们在真实试验数据集上对所提出的方法进行了评估,准确率达到了 87.81%。结果表明,MSF 和 MHA 模型可以提高欺骗检测任务的准确率。对比实验证明了我们提出的方法的有效性。
本文章由计算机程序翻译,如有差异,请以英文原文为准。

Deception detection with multi-scale feature and multi-head attention in videos

Deception detection with multi-scale feature and multi-head attention in videos

Detecting deception in videos has been a challenging task, especially in real world situations. In this study, we extracted the facial action units from the micro-expression, and then calculated the frequency and the number of occurrences of each action unit. To get more information on different scales, we proposed a combination scheme of Multi-Scale Feature (MSF) model and Multi-Head Attention (MHA). The MSF model consists of two CNN with different convolution kernels and GELU is used as the active function. The MHA model was designed to divide the input features into different subspaces and generate attention for each subspace to make the features more effective. We evaluated our proposed method on the Real-life Trial dataset and achieved an accuracy of 87.81%. The results show that the MSF and MHA model could increase the accuracy of deception detection task. And the comparative experiment demonstrates the effectiveness of our proposed method.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Multimedia Tools and Applications
Multimedia Tools and Applications 工程技术-工程:电子与电气
CiteScore
7.20
自引率
16.70%
发文量
2439
审稿时长
9.2 months
期刊介绍: Multimedia Tools and Applications publishes original research articles on multimedia development and system support tools as well as case studies of multimedia applications. It also features experimental and survey articles. The journal is intended for academics, practitioners, scientists and engineers who are involved in multimedia system research, design and applications. All papers are peer reviewed. Specific areas of interest include: - Multimedia Tools: - Multimedia Applications: - Prototype multimedia systems and platforms
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信