行动质量评估的层次神经符号法

Lauren Okamoto, Paritosh Parmar
{"title":"行动质量评估的层次神经符号法","authors":"Lauren Okamoto, Paritosh Parmar","doi":"arxiv-2403.13798","DOIUrl":null,"url":null,"abstract":"Action quality assessment (AQA) applies computer vision to quantitatively\nassess the performance or execution of a human action. Current AQA approaches\nare end-to-end neural models, which lack transparency and tend to be biased\nbecause they are trained on subjective human judgements as ground-truth. To\naddress these issues, we introduce a neuro-symbolic paradigm for AQA, which\nuses neural networks to abstract interpretable symbols from video data and\nmakes quality assessments by applying rules to those symbols. We take diving as\nthe case study. We found that domain experts prefer our system and find it more\ninformative than purely neural approaches to AQA in diving. Our system also\nachieves state-of-the-art action recognition and temporal segmentation, and\nautomatically generates a detailed report that breaks the dive down into its\nelements and provides objective scoring with visual evidence. As verified by a\ngroup of domain experts, this report may be used to assist judges in scoring,\nhelp train judges, and provide feedback to divers. We will open-source all of\nour annotated training data and code for ease of reproducibility.","PeriodicalId":501033,"journal":{"name":"arXiv - CS - Symbolic Computation","volume":"141 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-03-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Hierarchical NeuroSymbolic Approach for Action Quality Assessment\",\"authors\":\"Lauren Okamoto, Paritosh Parmar\",\"doi\":\"arxiv-2403.13798\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Action quality assessment (AQA) applies computer vision to quantitatively\\nassess the performance or execution of a human action. Current AQA approaches\\nare end-to-end neural models, which lack transparency and tend to be biased\\nbecause they are trained on subjective human judgements as ground-truth. To\\naddress these issues, we introduce a neuro-symbolic paradigm for AQA, which\\nuses neural networks to abstract interpretable symbols from video data and\\nmakes quality assessments by applying rules to those symbols. We take diving as\\nthe case study. We found that domain experts prefer our system and find it more\\ninformative than purely neural approaches to AQA in diving. Our system also\\nachieves state-of-the-art action recognition and temporal segmentation, and\\nautomatically generates a detailed report that breaks the dive down into its\\nelements and provides objective scoring with visual evidence. As verified by a\\ngroup of domain experts, this report may be used to assist judges in scoring,\\nhelp train judges, and provide feedback to divers. We will open-source all of\\nour annotated training data and code for ease of reproducibility.\",\"PeriodicalId\":501033,\"journal\":{\"name\":\"arXiv - CS - Symbolic Computation\",\"volume\":\"141 1\",\"pages\":\"\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-03-20\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"arXiv - CS - Symbolic Computation\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/arxiv-2403.13798\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - CS - Symbolic Computation","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2403.13798","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

摘要

动作质量评估(AQA)应用计算机视觉来定量评估人类动作的性能或执行情况。目前的 AQA 方法是端到端的神经模型,这种方法缺乏透明度,而且往往会产生偏差,因为它们是根据人类的主观判断作为基本真相进行训练的。为了解决这些问题,我们为 AQA 引入了神经符号范式,利用神经网络从视频数据中抽象出可解释的符号,并通过对这些符号应用规则来进行质量评估。我们以潜水为例进行研究。我们发现,领域专家更喜欢我们的系统,并认为它比纯粹的神经方法更有信息量。我们的系统还实现了最先进的动作识别和时间分割,并自动生成一份详细报告,将潜水分解为各个要素,并提供客观的视觉证据评分。经一组领域专家验证,该报告可用于协助评委评分、帮助培训评委并向潜水员提供反馈。我们将开源所有注释过的训练数据和代码,以方便复制。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Hierarchical NeuroSymbolic Approach for Action Quality Assessment
Action quality assessment (AQA) applies computer vision to quantitatively assess the performance or execution of a human action. Current AQA approaches are end-to-end neural models, which lack transparency and tend to be biased because they are trained on subjective human judgements as ground-truth. To address these issues, we introduce a neuro-symbolic paradigm for AQA, which uses neural networks to abstract interpretable symbols from video data and makes quality assessments by applying rules to those symbols. We take diving as the case study. We found that domain experts prefer our system and find it more informative than purely neural approaches to AQA in diving. Our system also achieves state-of-the-art action recognition and temporal segmentation, and automatically generates a detailed report that breaks the dive down into its elements and provides objective scoring with visual evidence. As verified by a group of domain experts, this report may be used to assist judges in scoring, help train judges, and provide feedback to divers. We will open-source all of our annotated training data and code for ease of reproducibility.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信