Mohammad T Elhakim, Sarah W Stougaard, Ole Graumann, Mads Nielsen, Oke Gerke, Lisbet B Larsen, Benjamin S B Rasmussen
求助PDF
{"title":"AI-integrated Screening to Replace Double Reading of Mammograms: A Population-wide Accuracy and Feasibility Study.","authors":"Mohammad T Elhakim, Sarah W Stougaard, Ole Graumann, Mads Nielsen, Oke Gerke, Lisbet B Larsen, Benjamin S B Rasmussen","doi":"10.1148/ryai.230529","DOIUrl":null,"url":null,"abstract":"<p><p><i>\"Just Accepted\" papers have undergone full peer review and have been accepted for publication in <i>Radiology: Artificial Intelligence</i>. This article will undergo copyediting, layout, and proof review before it is published in its final version. Please note that during production of the final copyedited article, errors may be discovered which could affect the content.</i> Mammography screening supported by deep learning-based artificial intelligence (AI) solutions can potentially reduce workload without compromising breast cancer detection accuracy, but the site of deployment in the workflow might be crucial. This retrospective study compared three simulated AI-integrated screening scenarios with standard double reading with arbitration in a sample of 249,402 mammograms from a representative screening population. A commercial AI system replaced the first reader (Scenario 1: Integrated AI<sub>first</sub>), the second reader (Scenario 2: Integrated AI<sub>second</sub>), or both readers for triaging of low- and high-risk cases (Integrated AI<sub>triage</sub>). AI threshold values were partly chosen based on previous validation and fixing screen-read volume reduction at approximately 50% across scenarios. Detection accuracy measures were calculated. Compared with standard double reading, Integrated AI<sub>first</sub> showed no evidence of a difference in accuracy metrics except for a higher arbitration rate (+0.99%; <i>P</i> < .001). Integrated AI<sub>second</sub> had lower sensitivity (-1.58%; <i>P</i> < 0.001), negative predictive value (NPV) (- 0.01%; <i>P</i> < .001) and recall rate (< 0.06%; <i>P</i> = 0.04), but a higher positive predictive value (PPV) (+0.03%; <i>P</i> < .001) and arbitration rate (+1.22%; <i>P</i> < .001). Integrated AI<sub>triage</sub> achieved higher sensitivity (+1.33%; <i>P</i> < .001), PPV (+0.36%; <i>P</i> = .03), and NPV (+0.01%; <i>P</i> < .001) but lower arbitration rate (-0.88%; <i>P</i> < .001). Replacing one or both readers with AI seems feasible, however, the site of application in the workflow can have clinically relevant effects on accuracy and workload. ©RSNA, 2024.</p>","PeriodicalId":29787,"journal":{"name":"Radiology-Artificial Intelligence","volume":null,"pages":null},"PeriodicalIF":8.1000,"publicationDate":"2024-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Radiology-Artificial Intelligence","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1148/ryai.230529","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0
引用
批量引用
Abstract
"Just Accepted" papers have undergone full peer review and have been accepted for publication in Radiology: Artificial Intelligence . This article will undergo copyediting, layout, and proof review before it is published in its final version. Please note that during production of the final copyedited article, errors may be discovered which could affect the content. Mammography screening supported by deep learning-based artificial intelligence (AI) solutions can potentially reduce workload without compromising breast cancer detection accuracy, but the site of deployment in the workflow might be crucial. This retrospective study compared three simulated AI-integrated screening scenarios with standard double reading with arbitration in a sample of 249,402 mammograms from a representative screening population. A commercial AI system replaced the first reader (Scenario 1: Integrated AIfirst ), the second reader (Scenario 2: Integrated AIsecond ), or both readers for triaging of low- and high-risk cases (Integrated AItriage ). AI threshold values were partly chosen based on previous validation and fixing screen-read volume reduction at approximately 50% across scenarios. Detection accuracy measures were calculated. Compared with standard double reading, Integrated AIfirst showed no evidence of a difference in accuracy metrics except for a higher arbitration rate (+0.99%; P < .001). Integrated AIsecond had lower sensitivity (-1.58%; P < 0.001), negative predictive value (NPV) (- 0.01%; P < .001) and recall rate (< 0.06%; P = 0.04), but a higher positive predictive value (PPV) (+0.03%; P < .001) and arbitration rate (+1.22%; P < .001). Integrated AItriage achieved higher sensitivity (+1.33%; P < .001), PPV (+0.36%; P = .03), and NPV (+0.01%; P < .001) but lower arbitration rate (-0.88%; P < .001). Replacing one or both readers with AI seems feasible, however, the site of application in the workflow can have clinically relevant effects on accuracy and workload. ©RSNA, 2024.
人工智能整合筛查取代乳房 X 光片双读:全人口准确性和可行性研究。
"刚刚接受 "的论文经过同行评审,已被接受在《放射学》上发表:人工智能》上发表。这篇文章在以最终版本发表之前,还将经过校对、排版和校对审核。请注意,在制作最终校对稿的过程中,可能会发现影响内容的错误。基于深度学习的人工智能(AI)解决方案支持的乳腺放射摄影筛查有可能在不影响乳腺癌检测准确性的情况下减少工作量,但工作流程中的部署地点可能至关重要。这项回顾性研究比较了三种模拟的人工智能集成筛查场景和标准双读与仲裁,样本来自具有代表性的筛查人群的 249,402 张乳房 X 光照片。商业人工智能系统取代了第一位读片员(情景 1:集成人工智能第一读片员)、第二位读片员(情景 2:集成人工智能第二读片员)或两位读片员,对低风险和高风险病例进行分流(集成人工智能分流)。人工智能阈值的部分选择是基于先前的验证,并将各种情况下的读屏量固定在大约 50%。计算了检测准确率。与标准双读相比,除了仲裁率较高(+0.99%;P < .001)外,综合人工智能第一在准确性指标上没有显示出差异。综合 AIsecond 的灵敏度 (-1.58%; P < 0.001)、阴性预测值 (NPV) (- 0.01%; P < 0.001) 和召回率 (< 0.06%; P = 0.04) 较低,但阳性预测值 (PPV) (+0.03%; P < 0.001) 和仲裁率 (+1.22%; P < 0.001) 较高。综合 AItriage 实现了更高的灵敏度(+1.33%;P < .001)、PPV(+0.36%;P = .03)和 NPV(+0.01%;P < .001),但仲裁率较低(-0.88%;P < .001)。用人工智能取代一台或两台读码器似乎是可行的,但工作流程中的应用位置会对准确性和工作量产生临床相关影响。©RSNA,2024。
本文章由计算机程序翻译,如有差异,请以英文原文为准。