“通过机器学习对人工智能驱动的视频分析进行微调;面瘫自动评估工具的开发

IF 3.2 2区 医学 Q1 SURGERY
Plastic and reconstructive surgery Pub Date : 2025-06-01 Epub Date: 2024-12-17 DOI:10.1097/PRS.0000000000011924
Takeichiro Kimura, Keigo Narita, Kohei Oyamada, Masahiko Ogura, Tomoyasu Ito, Takashi Okada, Akihiko Takushima
{"title":"“通过机器学习对人工智能驱动的视频分析进行微调;面瘫自动评估工具的开发","authors":"Takeichiro Kimura, Keigo Narita, Kohei Oyamada, Masahiko Ogura, Tomoyasu Ito, Takashi Okada, Akihiko Takushima","doi":"10.1097/PRS.0000000000011924","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>Establishment of a quantitative, objective evaluation tool for facial palsy has been a challenging issue for clinicians and researchers, and artificial intelligence-driven video analysis can be considered a reasonable solution. The authors introduced facial keypoint detection, which detects facial landmarks with 68 points, but existing models had been organized almost solely with images of healthy individuals, and low accuracy was presumed in the prediction of asymmetric faces of patients with facial palsy. The accuracy of the existing model was assessed by applying it to videos of 30 patients with facial palsy. Qualitative review clearly showed its insufficiency. The model was prone to detect patients' faces as symmetric, and was unable to detect eye closure. Thus, the authors enhanced the model through the machine-learning process of annotation (ie, fine-tuning).</p><p><strong>Methods: </strong>A total of 1181 images extracted from the videos of 196 patients were enrolled in the training, and these images underwent manual correction of 68 keypoints. The annotated data were integrated into the previous model with a stack of 2 hourglass networks combined with channel aggregation block.</p><p><strong>Results: </strong>The postannotation model showed improvement in normalized mean error from 0.026 to 0.018, and qualitative keypoint detection on each facial unit revealed improvements.</p><p><strong>Conclusions: </strong>Strict control of inter- and intra-annotator variability successfully fine-tuned the presented model. The new model is a promising solution for objective assessment of facial palsy.</p>","PeriodicalId":20128,"journal":{"name":"Plastic and reconstructive surgery","volume":" ","pages":"1071e-1081e"},"PeriodicalIF":3.2000,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Fine-Tuning on AI-Driven Video Analysis through Machine Learning: Development of an Automated Evaluation Tool of Facial Palsy.\",\"authors\":\"Takeichiro Kimura, Keigo Narita, Kohei Oyamada, Masahiko Ogura, Tomoyasu Ito, Takashi Okada, Akihiko Takushima\",\"doi\":\"10.1097/PRS.0000000000011924\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>Background: </strong>Establishment of a quantitative, objective evaluation tool for facial palsy has been a challenging issue for clinicians and researchers, and artificial intelligence-driven video analysis can be considered a reasonable solution. The authors introduced facial keypoint detection, which detects facial landmarks with 68 points, but existing models had been organized almost solely with images of healthy individuals, and low accuracy was presumed in the prediction of asymmetric faces of patients with facial palsy. The accuracy of the existing model was assessed by applying it to videos of 30 patients with facial palsy. Qualitative review clearly showed its insufficiency. The model was prone to detect patients' faces as symmetric, and was unable to detect eye closure. Thus, the authors enhanced the model through the machine-learning process of annotation (ie, fine-tuning).</p><p><strong>Methods: </strong>A total of 1181 images extracted from the videos of 196 patients were enrolled in the training, and these images underwent manual correction of 68 keypoints. The annotated data were integrated into the previous model with a stack of 2 hourglass networks combined with channel aggregation block.</p><p><strong>Results: </strong>The postannotation model showed improvement in normalized mean error from 0.026 to 0.018, and qualitative keypoint detection on each facial unit revealed improvements.</p><p><strong>Conclusions: </strong>Strict control of inter- and intra-annotator variability successfully fine-tuned the presented model. The new model is a promising solution for objective assessment of facial palsy.</p>\",\"PeriodicalId\":20128,\"journal\":{\"name\":\"Plastic and reconstructive surgery\",\"volume\":\" \",\"pages\":\"1071e-1081e\"},\"PeriodicalIF\":3.2000,\"publicationDate\":\"2025-06-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Plastic and reconstructive surgery\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1097/PRS.0000000000011924\",\"RegionNum\":2,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"2024/12/17 0:00:00\",\"PubModel\":\"Epub\",\"JCR\":\"Q1\",\"JCRName\":\"SURGERY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Plastic and reconstructive surgery","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1097/PRS.0000000000011924","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/12/17 0:00:00","PubModel":"Epub","JCR":"Q1","JCRName":"SURGERY","Score":null,"Total":0}
引用次数: 0

摘要

背景:建立定量、客观的面瘫评估工具一直是临床医生和研究人员面临的一大问题,在人工智能时代,人工智能驱动的视频分析可以被认为是解决这一长期争论的问题的合理方案。我们引入了人脸关键点检测,以68个点检测面部标志,但现有的模型几乎完全是用健康个体的图像组织的,在预测面瘫患者的不对称面部时准确率很低。将已有模型应用于30例面瘫患者的影像,对模型的准确性进行评价。质性回顾清楚地显示出其不足;容易察觉病人的脸是对称的,无法察觉闭上眼睛。因此,我们决定通过标注(微调)的机器学习过程来增强模型。方法:从196例患者的电影中提取1181幅图像进行训练,对这些图像进行68个关键点的人工校正。通过两个沙漏网络和通道聚合块的叠加,将标注的数据集成到之前的模型中。结果:注释后模型的归一化平均误差从0.026提高到0.018,每个面部单元的定性关键点检测也有所改善。结论:我们严格控制了注释者之间和注释者内部的可变性,成功地对呈现模型进行了微调,我们认为我们的新模型是客观评估面瘫的有希望的解决方案之一。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Fine-Tuning on AI-Driven Video Analysis through Machine Learning: Development of an Automated Evaluation Tool of Facial Palsy.

Background: Establishment of a quantitative, objective evaluation tool for facial palsy has been a challenging issue for clinicians and researchers, and artificial intelligence-driven video analysis can be considered a reasonable solution. The authors introduced facial keypoint detection, which detects facial landmarks with 68 points, but existing models had been organized almost solely with images of healthy individuals, and low accuracy was presumed in the prediction of asymmetric faces of patients with facial palsy. The accuracy of the existing model was assessed by applying it to videos of 30 patients with facial palsy. Qualitative review clearly showed its insufficiency. The model was prone to detect patients' faces as symmetric, and was unable to detect eye closure. Thus, the authors enhanced the model through the machine-learning process of annotation (ie, fine-tuning).

Methods: A total of 1181 images extracted from the videos of 196 patients were enrolled in the training, and these images underwent manual correction of 68 keypoints. The annotated data were integrated into the previous model with a stack of 2 hourglass networks combined with channel aggregation block.

Results: The postannotation model showed improvement in normalized mean error from 0.026 to 0.018, and qualitative keypoint detection on each facial unit revealed improvements.

Conclusions: Strict control of inter- and intra-annotator variability successfully fine-tuned the presented model. The new model is a promising solution for objective assessment of facial palsy.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
CiteScore
5.00
自引率
13.90%
发文量
1436
审稿时长
1.5 months
期刊介绍: For more than 70 years Plastic and Reconstructive Surgery® has been the one consistently excellent reference for every specialist who uses plastic surgery techniques or works in conjunction with a plastic surgeon. Plastic and Reconstructive Surgery® , the official journal of the American Society of Plastic Surgeons, is a benefit of Society membership, and is also available on a subscription basis. Plastic and Reconstructive Surgery® brings subscribers up-to-the-minute reports on the latest techniques and follow-up for all areas of plastic and reconstructive surgery, including breast reconstruction, experimental studies, maxillofacial reconstruction, hand and microsurgery, burn repair, cosmetic surgery, as well as news on medicolegal issues. The cosmetic section provides expanded coverage on new procedures and techniques and offers more cosmetic-specific content than any other journal. All subscribers enjoy full access to the Journal''s website, which features broadcast quality videos of reconstructive and cosmetic procedures, podcasts, comprehensive article archives dating to 1946, and additional benefits offered by the newly-redesigned website.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信