Effect of Artificial Intelligence as a Second Reader on the Lung Nodule Detection and Localization Accuracy of Radiologists and Non-radiology Physicians in Chest Radiographs: A Multicenter Reader Study.
IF 3.8 2区 医学Q1 RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING
Dennis Robert, Saigopal Sathyamurthy, Anshul Kumar Singh, Sri Anusha Matta, Manoj Tadepalli, Swetha Tanamala, Vijay Bosemani, Joseph Mammarappallil, Bunty Kundnani
{"title":"Effect of Artificial Intelligence as a Second Reader on the Lung Nodule Detection and Localization Accuracy of Radiologists and Non-radiology Physicians in Chest Radiographs: A Multicenter Reader Study.","authors":"Dennis Robert, Saigopal Sathyamurthy, Anshul Kumar Singh, Sri Anusha Matta, Manoj Tadepalli, Swetha Tanamala, Vijay Bosemani, Joseph Mammarappallil, Bunty Kundnani","doi":"10.1016/j.acra.2024.11.003","DOIUrl":null,"url":null,"abstract":"<p><strong>Rationale and objectives: </strong>Missed nodules in chest radiographs (CXRs) are common occurrences. We assessed the effect of artificial intelligence (AI) as a second reader on the accuracy of radiologists and non-radiology physicians in lung nodule detection and localization in CXRs.</p><p><strong>Materials and methods: </strong>This retrospective study using the multi-reader multi-case design included 300 CXRs acquired from 40 hospitals across the US. All CXRs had a paired follow-up image (chest CT or CXR) to augment the ground truth establishment for the presence and location of nodules on CXRs by five independent thoracic radiologists. 15 readers (nine radiologists and six non-radiology physicians) read each CXR twice in a second-reader paradigm, once without AI and then immediately with AI assistance. The primary analysis assessed the difference in area-under-the-alternative-free-response-receiver-operating-characteristic-curve (AFROC) of readers with and without AI. Case-level area-under-the-receiver-operating-characteristic-curve (AUROC), sensitivity, and specificity were assessed in secondary analyses.</p><p><strong>Results: </strong>A total of 300 CXRs (147 with nodules, 153 without nodules) from 300 patients (mean age, 64 years ± 15 [standard deviation]; 174 women) were included. The mean AFROC of readers was 0.73 without AI and 0.81 with AI (95% CI of difference, 0.05-0.10). Case-level AUROC was 0.77 without AI and 0.84 with AI (95% CI of difference, 0.04-0.09). Case-level sensitivity was 72.8% and 83.5% (95% CI of difference, 6.8-14.6) and specificity was 71.1% and 72.0% (95% CI of difference, -0.8-2.6) without and with AI, respectively.</p><p><strong>Conclusion: </strong>Using AI, readers detected and localized more nodules without any significant difference in false positive interpretations.</p>","PeriodicalId":50928,"journal":{"name":"Academic Radiology","volume":" ","pages":""},"PeriodicalIF":3.8000,"publicationDate":"2024-11-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Academic Radiology","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1016/j.acra.2024.11.003","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING","Score":null,"Total":0}
引用次数: 0
Abstract
Rationale and objectives: Missed nodules in chest radiographs (CXRs) are common occurrences. We assessed the effect of artificial intelligence (AI) as a second reader on the accuracy of radiologists and non-radiology physicians in lung nodule detection and localization in CXRs.
Materials and methods: This retrospective study using the multi-reader multi-case design included 300 CXRs acquired from 40 hospitals across the US. All CXRs had a paired follow-up image (chest CT or CXR) to augment the ground truth establishment for the presence and location of nodules on CXRs by five independent thoracic radiologists. 15 readers (nine radiologists and six non-radiology physicians) read each CXR twice in a second-reader paradigm, once without AI and then immediately with AI assistance. The primary analysis assessed the difference in area-under-the-alternative-free-response-receiver-operating-characteristic-curve (AFROC) of readers with and without AI. Case-level area-under-the-receiver-operating-characteristic-curve (AUROC), sensitivity, and specificity were assessed in secondary analyses.
Results: A total of 300 CXRs (147 with nodules, 153 without nodules) from 300 patients (mean age, 64 years ± 15 [standard deviation]; 174 women) were included. The mean AFROC of readers was 0.73 without AI and 0.81 with AI (95% CI of difference, 0.05-0.10). Case-level AUROC was 0.77 without AI and 0.84 with AI (95% CI of difference, 0.04-0.09). Case-level sensitivity was 72.8% and 83.5% (95% CI of difference, 6.8-14.6) and specificity was 71.1% and 72.0% (95% CI of difference, -0.8-2.6) without and with AI, respectively.
Conclusion: Using AI, readers detected and localized more nodules without any significant difference in false positive interpretations.
期刊介绍：
Academic Radiology publishes original reports of clinical and laboratory investigations in diagnostic imaging, the diagnostic use of radioactive isotopes, computed tomography, positron emission tomography, magnetic resonance imaging, ultrasound, digital subtraction angiography, image-guided interventions and related techniques. It also includes brief technical reports describing original observations, techniques, and instrumental developments; state-of-the-art reports on clinical issues, new technology and other topics of current medical importance; meta-analyses; scientific studies and opinions on radiologic education; and letters to the Editor.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
