Improving breast ultrasonography education: the impact of AI-based decision support on the performance of non-specialist medical professionals.

IF 2.4 3区医学 Q2 RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING

Ultrasonography Pub Date : 2025-03-01 Epub Date: 2024-12-12 DOI:10.14366/usg.24171

Sangwon Lee, Hye Sun Lee, Eunju Lee, Won Hwa Kim, Jaeil Kim, Jung Hyun Yoon

{"title":"Improving breast ultrasonography education: the impact of AI-based decision support on the performance of non-specialist medical professionals.","authors":"Sangwon Lee, Hye Sun Lee, Eunju Lee, Won Hwa Kim, Jaeil Kim, Jung Hyun Yoon","doi":"10.14366/usg.24171","DOIUrl":null,"url":null,"abstract":"Purpose: This study evaluated the educational impact of an artificial intelligence (AI)-based decision support system for breast ultrasonography (US) on medical professionals not specialized in breast imaging.Methods: In this multi-case, multi-reader study, educational materials, including American College of Radiology Breast Imaging Reporting and Data System (BI-RADS) descriptors, were provided alongside corresponding AI results during training. The AI system presented results in the form of AIheatmaps, AI scores, and AI-provided BI-RADS assessment categories. Forty-two readers evaluated the test set in three sessions: the first session (S1) occurred before the educational intervention, the second session (S2) followed education without AI assistance, and the third session (S3) took place after education with AI assistance. The area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and overall performance, were compared between the sessions.Results: The mean sensitivity increased from 66.5% (95% confidence interval [CI], 59.2% to 73.7%) to 88.7% (95% CI, 84.1% to 93.3%), with a statistically significant difference (P<0.001), and the AUC non-significantly increased from 0.664 (95% CI, 0.606 to 0.723) to 0.684 (95% CI, 0.620 to 0.748) (P=0.300). Both measures were higher in S2 than in S1. The AI-achieved AUC was comparable to that of the expert reader (0.747 [95% CI, 0.640 to 0.855] vs. 0.803 [95% CI, 0.706 to 0.900], P=0.217). Additionally, with AI assistance, the mean AUC for inexperienced readers was not significantly different from that of the expert reader (0.745 [95% CI, 0.660 to 0.830] vs. 0.803 [95% CI, 0.706 to 0.900], P=0.120).Conclusion: The mean AUC and sensitivity improved after incorporating AI into breast US education and interpretation. AI systems with high-level performance for breast US can potentially be used as educational tools in the interpretation of breast US images.","PeriodicalId":54227,"journal":{"name":"Ultrasonography","volume":" ","pages":"124-133"},"PeriodicalIF":2.4000,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11938798/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Ultrasonography","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.14366/usg.24171","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/12/12 0:00:00","PubModel":"Epub","JCR":"Q2","JCRName":"RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING","Score":null,"Total":0}

引用次数: 0

Abstract

Purpose: This study evaluated the educational impact of an artificial intelligence (AI)-based decision support system for breast ultrasonography (US) on medical professionals not specialized in breast imaging.

Methods: In this multi-case, multi-reader study, educational materials, including American College of Radiology Breast Imaging Reporting and Data System (BI-RADS) descriptors, were provided alongside corresponding AI results during training. The AI system presented results in the form of AIheatmaps, AI scores, and AI-provided BI-RADS assessment categories. Forty-two readers evaluated the test set in three sessions: the first session (S1) occurred before the educational intervention, the second session (S2) followed education without AI assistance, and the third session (S3) took place after education with AI assistance. The area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and overall performance, were compared between the sessions.

Results: The mean sensitivity increased from 66.5% (95% confidence interval [CI], 59.2% to 73.7%) to 88.7% (95% CI, 84.1% to 93.3%), with a statistically significant difference (P<0.001), and the AUC non-significantly increased from 0.664 (95% CI, 0.606 to 0.723) to 0.684 (95% CI, 0.620 to 0.748) (P=0.300). Both measures were higher in S2 than in S1. The AI-achieved AUC was comparable to that of the expert reader (0.747 [95% CI, 0.640 to 0.855] vs. 0.803 [95% CI, 0.706 to 0.900], P=0.217). Additionally, with AI assistance, the mean AUC for inexperienced readers was not significantly different from that of the expert reader (0.745 [95% CI, 0.660 to 0.830] vs. 0.803 [95% CI, 0.706 to 0.900], P=0.120).

Conclusion: The mean AUC and sensitivity improved after incorporating AI into breast US education and interpretation. AI systems with high-level performance for breast US can potentially be used as educational tools in the interpretation of breast US images.

查看原文本刊更多论文

求助全文

约1分钟内获得全文求助全文

来源期刊

Ultrasonography Medicine-Radiology, Nuclear Medicine and Imaging

CiteScore

5.10

自引率

6.50%

发文量

审稿时长

15 weeks

期刊介绍： Ultrasonography, the official English-language journal of the Korean Society of Ultrasound in Medicine (KSUM), is an international peer-reviewed academic journal dedicated to practice, research, technology, and education dealing with medical ultrasound. It is renamed from the Journal of Korean Society of Ultrasound in Medicine in January 2014, and published four times per year: January 1, April 1, July 1, and October 1. Original articles, technical notes, topical reviews, perspectives, pictorial essays, and timely editorial materials are published in Ultrasonography covering state-of-the-art content. Ultrasonography aims to provide updated information on new diagnostic concepts and technical developments, including experimental animal studies using new equipment in addition to well-designed reviews of contemporary issues in patient care. Along with running KSUM Open, the annual international congress of KSUM, Ultrasonography also serves as a medium for cooperation among physicians and specialists from around the world who are focusing on various ultrasound technology and disease problems and relevant basic science.