下载PDF
{"title":"Deep Learning to Detect Intracranial Hemorrhage in a National Teleradiology Program and the Impact on Interpretation Time.","authors":"Andrew James Del Gaizo, Thomas F Osborne, Troy Shahoumian, Robert Sherrier","doi":"10.1148/ryai.240067","DOIUrl":null,"url":null,"abstract":"<p><p>The diagnostic performance of an artificial intelligence (AI) clinical decision support solution for acute intracranial hemorrhage (ICH) detection was assessed in a large teleradiology practice. The impact on radiologist read times and system efficiency was also quantified. A total of 61 704 consecutive noncontrast head CT examinations were retrospectively evaluated. System performance was calculated along with mean and median read times for CT studies obtained before (baseline, pre-AI period; August 2021 to May 2022) and after (post-AI period; January 2023 to February 2024) AI implementation. The AI solution had a sensitivity of 75.6%, specificity of 92.1%, accuracy of 91.7%, prevalence of 2.70%, and positive predictive value of 21.1%. Of the 56 745 post-AI CT scans with no bleed identified by a radiologist, examinations falsely flagged as suspected ICH by the AI solution (<i>n</i> = 4464) took an average of 9 minutes 40 seconds (median, 8 minutes 7 seconds) to interpret as compared with 8 minutes 25 seconds (median, 6 minutes 48 seconds) for unremarkable CT scans before AI (<i>n</i> = 49 007) (<i>P</i> < .001) and 8 minutes 38 seconds (median, 6 minutes 53 seconds) after AI when ICH was not suspected by the AI solution (<i>n</i> = 52 281) (<i>P</i> < .001). CT scans with no bleed identified by the AI but reported as positive for ICH by the radiologist (<i>n</i> = 384) took an average of 14 minutes 23 seconds (median, 13 minutes 35 seconds) to interpret as compared with 13 minutes 34 seconds (median, 12 minutes 30 seconds) for CT scans correctly reported as a bleed by the AI (<i>n</i> = 1192) (<i>P</i> = .04). With lengthened read times for falsely flagged examinations, system inefficiencies may outweigh the potential benefits of using the tool in a high volume, low prevalence environment. <b>Keywords:</b> Artificial Intelligence, Intracranial Hemorrhage, Read Time, Report Turnaround Time, System Efficiency <i>Supplemental material is available for this article.</i> © RSNA, 2024.</p>","PeriodicalId":29787,"journal":{"name":"Radiology-Artificial Intelligence","volume":null,"pages":null},"PeriodicalIF":8.1000,"publicationDate":"2024-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11427938/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Radiology-Artificial Intelligence","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1148/ryai.240067","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0
引用
批量引用
Abstract
The diagnostic performance of an artificial intelligence (AI) clinical decision support solution for acute intracranial hemorrhage (ICH) detection was assessed in a large teleradiology practice. The impact on radiologist read times and system efficiency was also quantified. A total of 61 704 consecutive noncontrast head CT examinations were retrospectively evaluated. System performance was calculated along with mean and median read times for CT studies obtained before (baseline, pre-AI period; August 2021 to May 2022) and after (post-AI period; January 2023 to February 2024) AI implementation. The AI solution had a sensitivity of 75.6%, specificity of 92.1%, accuracy of 91.7%, prevalence of 2.70%, and positive predictive value of 21.1%. Of the 56 745 post-AI CT scans with no bleed identified by a radiologist, examinations falsely flagged as suspected ICH by the AI solution (n = 4464) took an average of 9 minutes 40 seconds (median, 8 minutes 7 seconds) to interpret as compared with 8 minutes 25 seconds (median, 6 minutes 48 seconds) for unremarkable CT scans before AI (n = 49 007) (P < .001) and 8 minutes 38 seconds (median, 6 minutes 53 seconds) after AI when ICH was not suspected by the AI solution (n = 52 281) (P < .001). CT scans with no bleed identified by the AI but reported as positive for ICH by the radiologist (n = 384) took an average of 14 minutes 23 seconds (median, 13 minutes 35 seconds) to interpret as compared with 13 minutes 34 seconds (median, 12 minutes 30 seconds) for CT scans correctly reported as a bleed by the AI (n = 1192) (P = .04). With lengthened read times for falsely flagged examinations, system inefficiencies may outweigh the potential benefits of using the tool in a high volume, low prevalence environment. Keywords: Artificial Intelligence, Intracranial Hemorrhage, Read Time, Report Turnaround Time, System Efficiency Supplemental material is available for this article. © RSNA, 2024.
深度学习检测国家远程放射学项目中的颅内出血及其对判读时间的影响。
"刚刚接受 "的论文经过同行评审,已被接受在《放射学》上发表:人工智能》上发表。这篇文章在以最终版本发表之前,还将经过校对、排版和校对审核。请注意,在制作最终校对稿的过程中,可能会发现一些可能影响内容的错误。在大型远程放射学实践中评估了人工智能(AI)临床决策支持(CDS)解决方案对急性颅内出血(ICH)检测的诊断性能。同时还量化了该方案对放射医师读片时间和系统效率的影响。共对 61,704 例连续的非对比头部 CT(NCHCT)进行了回顾性评估。计算了系统性能以及人工智能前(基线:2021 年 8 月至 2022 年 5 月)和人工智能后(2023 年 1 月至 2024 年 2 月)NCHCT 的平均和中位读取时间值。人工智能解决方案的灵敏度为 75.6%,特异性为 92.1%,准确性为 91.7%,流行率为 2.70%,阳性预测值为 21.1%。在56,745例经放射科医生确认无出血的AI后NCHCT中,被AI解决方案误标记为疑似ICH的检查(n = 4,464)的平均判读时间为9分40秒/中位数为8分7秒,而AI前无异常NCHCT(n = 49,007)的平均判读时间为8分25秒/中位数为6分48秒(P < .001)和 AI 后平均 8 分 38 秒/6 分 53 秒中位数(当 AI 方案未怀疑 ICH 时,n = 52,281 )(P < .001 )。人工智能未识别出血但放射科医生报告为 ICH 阳性的 NCHCT(n = 384)平均判读时间为 14 分 23 秒/中位数为 13 分 35 秒,而人工智能正确报告为出血的 NCHCT(n = 1192)平均判读时间为 13 分 34 秒/中位数为 12 分 30 秒(P = .04)。由于错误标记检查的读取时间延长,系统的低效率可能会超过在高流量、低流行率环境中使用该工具的潜在益处。©RSNA,2024。
本文章由计算机程序翻译,如有差异,请以英文原文为准。