Mi-ri Kwon, Yoosoo Chang, Soo-Youn Ham, Yoosun Cho, Eun Young Kim, Jeonggyu Kang, Eun Kyung Park, Ki Hwan Kim, Minjeong Kim, Tae Soo Kim, Hyeonsoo Lee, Ria Kwon, Ga-Young Lim, Hye Rin Choi, JunHyeok Choi, Shin Ho Kook, Seungho Ryu
{"title":"根据乳腺密度进行乳腺 X 射线筛查的效果:放射科医生与独立智能检测之间的比较","authors":"Mi-ri Kwon, Yoosoo Chang, Soo-Youn Ham, Yoosun Cho, Eun Young Kim, Jeonggyu Kang, Eun Kyung Park, Ki Hwan Kim, Minjeong Kim, Tae Soo Kim, Hyeonsoo Lee, Ria Kwon, Ga-Young Lim, Hye Rin Choi, JunHyeok Choi, Shin Ho Kook, Seungho Ryu","doi":"10.1186/s13058-024-01821-w","DOIUrl":null,"url":null,"abstract":"Artificial intelligence (AI) algorithms for the independent assessment of screening mammograms have not been well established in a large screening cohort of Asian women. We compared the performance of screening digital mammography considering breast density, between radiologists and AI standalone detection among Korean women. We retrospectively included 89,855 Korean women who underwent their initial screening digital mammography from 2009 to 2020. Breast cancer within 12 months of the screening mammography was the reference standard, according to the National Cancer Registry. Lunit software was used to determine the probability of malignancy scores, with a cutoff of 10% for breast cancer detection. The AI’s performance was compared with that of the final Breast Imaging Reporting and Data System category, as recorded by breast radiologists. Breast density was classified into four categories (A–D) based on the radiologist and AI-based assessments. The performance metrics (cancer detection rate [CDR], sensitivity, specificity, positive predictive value [PPV], recall rate, and area under the receiver operating characteristic curve [AUC]) were compared across breast density categories. Mean participant age was 43.5 ± 8.7 years; 143 breast cancer cases were identified within 12 months. The CDRs (1.1/1000 examination) and sensitivity values showed no significant differences between radiologist and AI-based results (69.9% [95% confidence interval [CI], 61.7–77.3] vs. 67.1% [95% CI, 58.8–74.8]). However, the AI algorithm showed better specificity (93.0% [95% CI, 92.9–93.2] vs. 77.6% [95% CI, 61.7–77.9]), PPV (1.5% [95% CI, 1.2–1.9] vs. 0.5% [95% CI, 0.4–0.6]), recall rate (7.1% [95% CI, 6.9–7.2] vs. 22.5% [95% CI, 22.2–22.7]), and AUC values (0.8 [95% CI, 0.76–0.84] vs. 0.74 [95% CI, 0.7–0.78]) (all P < 0.05). Radiologist and AI-based results showed the best performance in the non-dense category; the CDR and sensitivity were higher for radiologists in the heterogeneously dense category (P = 0.059). However, the specificity, PPV, and recall rate consistently favored AI-based results across all categories, including the extremely dense category. AI-based software showed slightly lower sensitivity, although the difference was not statistically significant. However, it outperformed radiologists in recall rate, specificity, PPV, and AUC, with disparities most prominent in extremely dense breast tissue.","PeriodicalId":9222,"journal":{"name":"Breast Cancer Research","volume":null,"pages":null},"PeriodicalIF":6.1000,"publicationDate":"2024-04-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Screening mammography performance according to breast density: a comparison between radiologists versus standalone intelligence detection\",\"authors\":\"Mi-ri Kwon, Yoosoo Chang, Soo-Youn Ham, Yoosun Cho, Eun Young Kim, Jeonggyu Kang, Eun Kyung Park, Ki Hwan Kim, Minjeong Kim, Tae Soo Kim, Hyeonsoo Lee, Ria Kwon, Ga-Young Lim, Hye Rin Choi, JunHyeok Choi, Shin Ho Kook, Seungho Ryu\",\"doi\":\"10.1186/s13058-024-01821-w\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Artificial intelligence (AI) algorithms for the independent assessment of screening mammograms have not been well established in a large screening cohort of Asian women. We compared the performance of screening digital mammography considering breast density, between radiologists and AI standalone detection among Korean women. We retrospectively included 89,855 Korean women who underwent their initial screening digital mammography from 2009 to 2020. Breast cancer within 12 months of the screening mammography was the reference standard, according to the National Cancer Registry. Lunit software was used to determine the probability of malignancy scores, with a cutoff of 10% for breast cancer detection. The AI’s performance was compared with that of the final Breast Imaging Reporting and Data System category, as recorded by breast radiologists. Breast density was classified into four categories (A–D) based on the radiologist and AI-based assessments. The performance metrics (cancer detection rate [CDR], sensitivity, specificity, positive predictive value [PPV], recall rate, and area under the receiver operating characteristic curve [AUC]) were compared across breast density categories. Mean participant age was 43.5 ± 8.7 years; 143 breast cancer cases were identified within 12 months. The CDRs (1.1/1000 examination) and sensitivity values showed no significant differences between radiologist and AI-based results (69.9% [95% confidence interval [CI], 61.7–77.3] vs. 67.1% [95% CI, 58.8–74.8]). However, the AI algorithm showed better specificity (93.0% [95% CI, 92.9–93.2] vs. 77.6% [95% CI, 61.7–77.9]), PPV (1.5% [95% CI, 1.2–1.9] vs. 0.5% [95% CI, 0.4–0.6]), recall rate (7.1% [95% CI, 6.9–7.2] vs. 22.5% [95% CI, 22.2–22.7]), and AUC values (0.8 [95% CI, 0.76–0.84] vs. 0.74 [95% CI, 0.7–0.78]) (all P < 0.05). Radiologist and AI-based results showed the best performance in the non-dense category; the CDR and sensitivity were higher for radiologists in the heterogeneously dense category (P = 0.059). However, the specificity, PPV, and recall rate consistently favored AI-based results across all categories, including the extremely dense category. AI-based software showed slightly lower sensitivity, although the difference was not statistically significant. However, it outperformed radiologists in recall rate, specificity, PPV, and AUC, with disparities most prominent in extremely dense breast tissue.\",\"PeriodicalId\":9222,\"journal\":{\"name\":\"Breast Cancer Research\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":6.1000,\"publicationDate\":\"2024-04-22\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Breast Cancer Research\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1186/s13058-024-01821-w\",\"RegionNum\":1,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"ONCOLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Breast Cancer Research","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1186/s13058-024-01821-w","RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ONCOLOGY","Score":null,"Total":0}
引用次数: 0
摘要
人工智能(AI)算法对乳腺X光筛查的独立评估尚未在亚洲妇女的大型筛查队列中得到很好的证实。我们比较了放射科医生和人工智能独立检测在韩国女性中考虑乳腺密度的筛查数字乳腺X光摄影的性能。我们回顾性地纳入了 2009 年至 2020 年期间接受初次数字乳腺 X 光摄影筛查的 89,855 名韩国女性。根据国家癌症登记处的数据,筛查乳房 X 射线照相术后 12 个月内的乳腺癌是参考标准。Lunit 软件用于确定恶性肿瘤概率分数,乳腺癌检测的临界值为 10%。根据乳腺放射科医生的记录,将人工智能的性能与乳腺成像报告和数据系统的最终分类进行了比较。根据放射科医生和人工智能的评估结果,乳腺密度被分为四类(A-D)。对不同乳腺密度类别的性能指标(癌症检出率[CDR]、灵敏度、特异性、阳性预测值[PPV]、召回率和接收者工作特征曲线下面积[AUC])进行了比较。参与者的平均年龄为 43.5 ± 8.7 岁,在 12 个月内发现了 143 例乳腺癌病例。CDR(1.1/1000 次检查)和灵敏度值显示,放射科医生和人工智能结果之间没有明显差异(69.9% [95% 置信区间 [CI], 61.7-77.3] vs. 67.1% [95% CI, 58.8-74.8])。不过,人工智能算法的特异性(93.0% [95% CI, 92.9-93.2] vs. 77.6% [95% CI, 61.7-77.9])、PPV(1.5% [95% CI, 1.2-1.9] vs. 0.5% [95% CI, 0.4-0.6])、召回率(7.1% [95% CI, 6.9-7.2] vs. 22.5% [95% CI, 22.2-22.7])和 AUC 值(0.8 [95% CI, 0.76-0.84] vs. 0.74 [95% CI, 0.7-0.78] )(所有 P <0.05)。放射科医生和基于人工智能的结果在非致密类别中表现最佳;在异质致密类别中,放射科医生的 CDR 和灵敏度更高(P = 0.059)。不过,在包括极度致密类别在内的所有类别中,基于人工智能的结果在特异性、PPV 和召回率方面始终更胜一筹。基于人工智能的软件灵敏度略低,但差异无统计学意义。不过,它在召回率、特异性、PPV 和 AUC 方面都优于放射科医生,在极致密乳腺组织方面差距最为明显。
Screening mammography performance according to breast density: a comparison between radiologists versus standalone intelligence detection
Artificial intelligence (AI) algorithms for the independent assessment of screening mammograms have not been well established in a large screening cohort of Asian women. We compared the performance of screening digital mammography considering breast density, between radiologists and AI standalone detection among Korean women. We retrospectively included 89,855 Korean women who underwent their initial screening digital mammography from 2009 to 2020. Breast cancer within 12 months of the screening mammography was the reference standard, according to the National Cancer Registry. Lunit software was used to determine the probability of malignancy scores, with a cutoff of 10% for breast cancer detection. The AI’s performance was compared with that of the final Breast Imaging Reporting and Data System category, as recorded by breast radiologists. Breast density was classified into four categories (A–D) based on the radiologist and AI-based assessments. The performance metrics (cancer detection rate [CDR], sensitivity, specificity, positive predictive value [PPV], recall rate, and area under the receiver operating characteristic curve [AUC]) were compared across breast density categories. Mean participant age was 43.5 ± 8.7 years; 143 breast cancer cases were identified within 12 months. The CDRs (1.1/1000 examination) and sensitivity values showed no significant differences between radiologist and AI-based results (69.9% [95% confidence interval [CI], 61.7–77.3] vs. 67.1% [95% CI, 58.8–74.8]). However, the AI algorithm showed better specificity (93.0% [95% CI, 92.9–93.2] vs. 77.6% [95% CI, 61.7–77.9]), PPV (1.5% [95% CI, 1.2–1.9] vs. 0.5% [95% CI, 0.4–0.6]), recall rate (7.1% [95% CI, 6.9–7.2] vs. 22.5% [95% CI, 22.2–22.7]), and AUC values (0.8 [95% CI, 0.76–0.84] vs. 0.74 [95% CI, 0.7–0.78]) (all P < 0.05). Radiologist and AI-based results showed the best performance in the non-dense category; the CDR and sensitivity were higher for radiologists in the heterogeneously dense category (P = 0.059). However, the specificity, PPV, and recall rate consistently favored AI-based results across all categories, including the extremely dense category. AI-based software showed slightly lower sensitivity, although the difference was not statistically significant. However, it outperformed radiologists in recall rate, specificity, PPV, and AUC, with disparities most prominent in extremely dense breast tissue.
期刊介绍:
Breast Cancer Research is an international, peer-reviewed online journal, publishing original research, reviews, editorials and reports. Open access research articles of exceptional interest are published in all areas of biology and medicine relevant to breast cancer, including normal mammary gland biology, with special emphasis on the genetic, biochemical, and cellular basis of breast cancer. In addition to basic research, the journal publishes preclinical, translational and clinical studies with a biological basis, including Phase I and Phase II trials.