超越一次性验证:基于预测和诊断的人工智能医疗设备的自适应验证框架

Florian Hellmeier, Kay Brosien, Carsten Eickhoff, Alexander Meyer
{"title":"超越一次性验证:基于预测和诊断的人工智能医疗设备的自适应验证框架","authors":"Florian Hellmeier, Kay Brosien, Carsten Eickhoff, Alexander Meyer","doi":"arxiv-2409.04794","DOIUrl":null,"url":null,"abstract":"Prognostic and diagnostic AI-based medical devices hold immense promise for\nadvancing healthcare, yet their rapid development has outpaced the\nestablishment of appropriate validation methods. Existing approaches often fall\nshort in addressing the complexity of practically deploying these devices and\nensuring their effective, continued operation in real-world settings. Building\non recent discussions around the validation of AI models in medicine and\ndrawing from validation practices in other fields, a framework to address this\ngap is presented. It offers a structured, robust approach to validation that\nhelps ensure device reliability across differing clinical environments. The\nprimary challenges to device performance upon deployment are discussed while\nhighlighting the impact of changes related to individual healthcare\ninstitutions and operational processes. The presented framework emphasizes the\nimportance of repeating validation and fine-tuning during deployment, aiming to\nmitigate these issues while being adaptable to challenges unforeseen during\ndevice development. The framework is also positioned within the current US and\nEU regulatory landscapes, underscoring its practical viability and relevance\nconsidering regulatory requirements. Additionally, a practical example\ndemonstrating potential benefits of the framework is presented. Lastly,\nguidance on assessing model performance is offered and the importance of\ninvolving clinical stakeholders in the validation and fine-tuning process is\ndiscussed.","PeriodicalId":501112,"journal":{"name":"arXiv - CS - Computers and Society","volume":"11 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Beyond One-Time Validation: A Framework for Adaptive Validation of Prognostic and Diagnostic AI-based Medical Devices\",\"authors\":\"Florian Hellmeier, Kay Brosien, Carsten Eickhoff, Alexander Meyer\",\"doi\":\"arxiv-2409.04794\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Prognostic and diagnostic AI-based medical devices hold immense promise for\\nadvancing healthcare, yet their rapid development has outpaced the\\nestablishment of appropriate validation methods. Existing approaches often fall\\nshort in addressing the complexity of practically deploying these devices and\\nensuring their effective, continued operation in real-world settings. Building\\non recent discussions around the validation of AI models in medicine and\\ndrawing from validation practices in other fields, a framework to address this\\ngap is presented. It offers a structured, robust approach to validation that\\nhelps ensure device reliability across differing clinical environments. The\\nprimary challenges to device performance upon deployment are discussed while\\nhighlighting the impact of changes related to individual healthcare\\ninstitutions and operational processes. The presented framework emphasizes the\\nimportance of repeating validation and fine-tuning during deployment, aiming to\\nmitigate these issues while being adaptable to challenges unforeseen during\\ndevice development. The framework is also positioned within the current US and\\nEU regulatory landscapes, underscoring its practical viability and relevance\\nconsidering regulatory requirements. Additionally, a practical example\\ndemonstrating potential benefits of the framework is presented. Lastly,\\nguidance on assessing model performance is offered and the importance of\\ninvolving clinical stakeholders in the validation and fine-tuning process is\\ndiscussed.\",\"PeriodicalId\":501112,\"journal\":{\"name\":\"arXiv - CS - Computers and Society\",\"volume\":\"11 1\",\"pages\":\"\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-09-07\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"arXiv - CS - Computers and Society\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/arxiv-2409.04794\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - CS - Computers and Society","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2409.04794","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

摘要

基于预测和诊断的人工智能医疗设备在促进医疗保健方面大有可为,但其快速发展的速度超过了适当验证方法的建立速度。现有的方法往往无法解决实际部署这些设备的复杂性,也无法确保它们在现实环境中有效、持续地运行。本文以最近围绕医学人工智能模型验证的讨论为基础,并借鉴了其他领域的验证实践,提出了一个解决这一差距的框架。它提供了一种结构化、稳健的验证方法,有助于确保设备在不同临床环境中的可靠性。文中讨论了设备部署后性能所面临的主要挑战,同时强调了与各个医疗机构和操作流程相关的变化所带来的影响。所提出的框架强调了在部署过程中重复验证和微调的重要性,旨在解决这些问题,同时适应设备开发过程中无法预见的挑战。该框架的定位也符合当前美国和欧盟的监管环境,强调了其在考虑监管要求时的实际可行性和相关性。此外,还介绍了一个实例,展示了该框架的潜在优势。最后,还提供了评估模型性能的指导,并讨论了让临床利益相关者参与验证和微调过程的重要性。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Beyond One-Time Validation: A Framework for Adaptive Validation of Prognostic and Diagnostic AI-based Medical Devices
Prognostic and diagnostic AI-based medical devices hold immense promise for advancing healthcare, yet their rapid development has outpaced the establishment of appropriate validation methods. Existing approaches often fall short in addressing the complexity of practically deploying these devices and ensuring their effective, continued operation in real-world settings. Building on recent discussions around the validation of AI models in medicine and drawing from validation practices in other fields, a framework to address this gap is presented. It offers a structured, robust approach to validation that helps ensure device reliability across differing clinical environments. The primary challenges to device performance upon deployment are discussed while highlighting the impact of changes related to individual healthcare institutions and operational processes. The presented framework emphasizes the importance of repeating validation and fine-tuning during deployment, aiming to mitigate these issues while being adaptable to challenges unforeseen during device development. The framework is also positioned within the current US and EU regulatory landscapes, underscoring its practical viability and relevance considering regulatory requirements. Additionally, a practical example demonstrating potential benefits of the framework is presented. Lastly, guidance on assessing model performance is offered and the importance of involving clinical stakeholders in the validation and fine-tuning process is discussed.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信