Diagnostic performance of commercial AI systems versus participating radiologists for pulmonary nodule detection in routine clinical practice.

IF 2.1 4区医学

Japanese Journal of Radiology Pub Date : 2026-10-01 Epub Date: 2026-06-18 DOI:10.1007/s11604-026-02029-y

Hiroaki Watanabe, Hiroyuki Morisaka, Kaori Aoyagi, Ryota Tozuka, Tihan Wumu, Takanori Ii, Kasane Sakamoto, Tomomi Yato, Shintaro Ichikawa

{"title":"Diagnostic performance of commercial AI systems versus participating radiologists for pulmonary nodule detection in routine clinical practice.","authors":"Hiroaki Watanabe, Hiroyuki Morisaka, Kaori Aoyagi, Ryota Tozuka, Tihan Wumu, Takanori Ii, Kasane Sakamoto, Tomomi Yato, Shintaro Ichikawa","doi":"10.1007/s11604-026-02029-y","DOIUrl":null,"url":null,"abstract":"Purpose: To evaluate the diagnostic performance of two commercial artificial intelligence (AI) systems versus that of radiologists on routine clinical chest computed tomography (CT) and to identify the imaging characteristics that limit AI performance.Materials and methods: We retrospectively analyzed the 5-mm-slice chest CT of 102 patients (353 nodules or masses). The detection performance of two board-certified radiologists and two commercial AI systems was compared against an expert-established reference standard. Sensitivity, false positives (FPs) per scan, and positive predictive value (PPV) were evaluated. Logistic regression identified factors influencing detection.Results: The radiologists demonstrated higher sensitivity (90.4-94.3%) than the AI systems (80.2-89.2%) and significantly fewer FPs per scan (1.61-3.56 vs. 5.28-7.97; p < 0.001), resulting in superior PPVs. Multivariate analysis revealed divergent limitations: radiologists were challenged by intrinsic features (e.g., ground-glass nodules [GGNs]), whereas AI performance was degraded by case-level complexity (multiple nodules), specific locations (central), and atypical morphologies (large masses).Conclusion: In this study, the participating radiologists outperformed the commercial AI systems in routine clinical settings. Both exhibited distinct weakness profiles. Current AI systems are best suited as complementary tools rather than autonomous readers, provided FPs are managed effectively.","PeriodicalId":14691,"journal":{"name":"Japanese Journal of Radiology","volume":" ","pages":"1779-1790"},"PeriodicalIF":2.1000,"publicationDate":"2026-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13624007/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Japanese Journal of Radiology","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1007/s11604-026-02029-y","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2026/6/18 0:00:00","PubModel":"Epub","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 0

Abstract

Purpose: To evaluate the diagnostic performance of two commercial artificial intelligence (AI) systems versus that of radiologists on routine clinical chest computed tomography (CT) and to identify the imaging characteristics that limit AI performance.

Materials and methods: We retrospectively analyzed the 5-mm-slice chest CT of 102 patients (353 nodules or masses). The detection performance of two board-certified radiologists and two commercial AI systems was compared against an expert-established reference standard. Sensitivity, false positives (FPs) per scan, and positive predictive value (PPV) were evaluated. Logistic regression identified factors influencing detection.

Results: The radiologists demonstrated higher sensitivity (90.4-94.3%) than the AI systems (80.2-89.2%) and significantly fewer FPs per scan (1.61-3.56 vs. 5.28-7.97; p < 0.001), resulting in superior PPVs. Multivariate analysis revealed divergent limitations: radiologists were challenged by intrinsic features (e.g., ground-glass nodules [GGNs]), whereas AI performance was degraded by case-level complexity (multiple nodules), specific locations (central), and atypical morphologies (large masses).

Conclusion: In this study, the participating radiologists outperformed the commercial AI systems in routine clinical settings. Both exhibited distinct weakness profiles. Current AI systems are best suited as complementary tools rather than autonomous readers, provided FPs are managed effectively.

查看原文本刊更多论文

在常规临床实践中，商业人工智能系统与参与的放射科医生在肺结节检测方面的诊断性能。

目的：评估两种商业人工智能（AI）系统与放射科医生在常规临床胸部计算机断层扫描（CT）上的诊断性能，并确定限制AI性能的成像特征。材料和方法：回顾性分析102例（353个结节或肿块）的5mm层胸部CT表现。两位委员会认证的放射科医生和两个商业人工智能系统的检测性能与专家建立的参考标准进行了比较。评估灵敏度、每次扫描的假阳性（FPs）和阳性预测值（PPV）。Logistic回归确定了影响检测的因素。结果：放射科医生的灵敏度（90.4-94.3%）高于人工智能系统（80.2-89.2%），且每次扫描的FPs显著低于人工智能系统（1.61-3.56 vs. 5.28-7.97）； p结论：在本研究中，参与研究的放射科医生在常规临床环境中的表现优于商业人工智能系统。两人都表现出明显的弱点。当前的AI系统最适合作为补充工具，而不是自主阅读器，前提是FPs得到有效管理。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Japanese Journal of Radiology Medicine-Radiology, Nuclear Medicine and Imaging

自引率

4.80%

发文量

133

期刊介绍： Japanese Journal of Radiology is a peer-reviewed journal, officially published by the Japan Radiological Society. The main purpose of the journal is to provide a forum for the publication of papers documenting recent advances and new developments in the field of radiology in medicine and biology. The scope of Japanese Journal of Radiology encompasses but is not restricted to diagnostic radiology, interventional radiology, radiation oncology, nuclear medicine, radiation physics, and radiation biology. Additionally, the journal covers technical and industrial innovations. The journal welcomes original articles, technical notes, review articles, pictorial essays and letters to the editor. The journal also provides announcements from the boards and the committees of the society. Membership in the Japan Radiological Society is not a prerequisite for submission. Contributions are welcomed from all parts of the world.