Retrospective evaluation of a CE-marked AI system, including 1,017,208 mammography screening examinations.

IF 4.7 2区医学 Q1 RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING

European Radiology Pub Date : 2025-03-26 DOI:10.1007/s00330-025-11521-4

Tone Hovda, Marthe Larsen, Marie Burns Bergan, Jonas Gjesvik, Lars A Akslen, Solveig Hofvind

{"title":"Retrospective evaluation of a CE-marked AI system, including 1,017,208 mammography screening examinations.","authors":"Tone Hovda, Marthe Larsen, Marie Burns Bergan, Jonas Gjesvik, Lars A Akslen, Solveig Hofvind","doi":"10.1007/s00330-025-11521-4","DOIUrl":null,"url":null,"abstract":"Objectives: To retrospectively evaluate the performance of a CE-marked AI system for identifying breast cancer on screening mammograms. Evidence from large retrospective studies is crucial for planning prospective studies and to further ensure safe implementation.Materials and methods: We used data from screening examinations performed from 2004 to 2021 at ten breast centers in BreastScreen Norway. In the standard independent double reading setting, each radiologist scored each breast from 1 (negative) to 5 (high probability of cancer). The AI system assigned each examination an NT and an SN score; the NT score aimed to classify examinations as negative with minimal misclassification while the SN score aimed to classify examinations as positive with high confidence. N70 was defined as being among the 70% with the lowest NT score and P3 was defined as being among the 3% with the highest SN score.Results: A total of 1,017,208 screening examinations were included in the study sample. At N70, 1.8% (107/5977) of the screen-detected and 34.5% (625/1812) of the interval cancers were defined as negative. Using P3 to define cases as positive, 81.5% (4871/5977) of the screen-detected and 19.0% (344/1812) of the interval cancers were defined as positive. Among the screen-detected cancers in N70, 11.2% (12/107) had an interpretation score > 2 by both radiologists.Conclusion: The AI system performed well according to identifying negative cases and cancer cases. Thus, the AI system can be used to reduce workload for the radiologists and potentially increase the sensitivity of mammography.Key points: Question Results from large mammography screening samples not used in training AI algorithms are important to consider when planning prospective studies and implementation. Findings More than 80% of the screening-detected cancers were classified as positive by AI when considering 3% of the examinations with the highest AI risk score as positive. Clinical relevance A lack of radiologists is a challenge in mammographic screening. Our findings support other studies that suggest the use of AI to reduce screen-reading workload.","PeriodicalId":12076,"journal":{"name":"European Radiology","volume":" ","pages":""},"PeriodicalIF":4.7000,"publicationDate":"2025-03-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"European Radiology","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1007/s00330-025-11521-4","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING","Score":null,"Total":0}

引用次数: 0

Abstract

Objectives: To retrospectively evaluate the performance of a CE-marked AI system for identifying breast cancer on screening mammograms. Evidence from large retrospective studies is crucial for planning prospective studies and to further ensure safe implementation.

Materials and methods: We used data from screening examinations performed from 2004 to 2021 at ten breast centers in BreastScreen Norway. In the standard independent double reading setting, each radiologist scored each breast from 1 (negative) to 5 (high probability of cancer). The AI system assigned each examination an NT and an SN score; the NT score aimed to classify examinations as negative with minimal misclassification while the SN score aimed to classify examinations as positive with high confidence. N70 was defined as being among the 70% with the lowest NT score and P3 was defined as being among the 3% with the highest SN score.

Results: A total of 1,017,208 screening examinations were included in the study sample. At N70, 1.8% (107/5977) of the screen-detected and 34.5% (625/1812) of the interval cancers were defined as negative. Using P3 to define cases as positive, 81.5% (4871/5977) of the screen-detected and 19.0% (344/1812) of the interval cancers were defined as positive. Among the screen-detected cancers in N70, 11.2% (12/107) had an interpretation score > 2 by both radiologists.

Conclusion: The AI system performed well according to identifying negative cases and cancer cases. Thus, the AI system can be used to reduce workload for the radiologists and potentially increase the sensitivity of mammography.

Key points: Question Results from large mammography screening samples not used in training AI algorithms are important to consider when planning prospective studies and implementation. Findings More than 80% of the screening-detected cancers were classified as positive by AI when considering 3% of the examinations with the highest AI risk score as positive. Clinical relevance A lack of radiologists is a challenge in mammographic screening. Our findings support other studies that suggest the use of AI to reduce screen-reading workload.

查看原文本刊更多论文

ce标记人工智能系统的回顾性评估，包括1,017,208次乳房x光筛查检查。

目的：回顾性评价ce标记的人工智能系统在筛查乳房x光检查中识别乳腺癌的性能。来自大型回顾性研究的证据对于规划前瞻性研究和进一步确保安全实施至关重要。材料和方法：我们使用了2004年至2021年在挪威乳房筛查中心进行的筛查检查的数据。在标准的独立双读设置中，每个放射科医生对每个乳房进行评分，从1（阴性）到5（高癌症可能性）。AI系统为每次考试分配NT和SN分数；NT评分旨在将检查分类为阴性，错误分类最少；SN评分旨在将检查分类为阳性，可信度高。N70定义为NT评分最低的70%，P3定义为SN评分最高的3%。结果：研究样本共纳入1017208例筛查检查。在N70时，1.8%（107/5977）的筛查检测和34.5%（625/1812）的间隔期癌症被定义为阴性。使用P3定义病例为阳性，81.5%（4871/5977）的筛检和19.0%（344/1812）的间期癌定义为阳性。在N70筛查发现的癌症中，11.2%（12/107）的两名放射科医生的解释评分为bb0.2。结论：人工智能系统在识别阴性病例和癌症病例方面表现良好。因此，人工智能系统可以用来减少放射科医生的工作量，并有可能提高乳房x光检查的灵敏度。在规划前瞻性研究和实施时，未用于训练AI算法的大型乳房x光检查样本的结果是重要的考虑因素。研究结果：考虑到人工智能风险评分最高的检查中有3%为阳性，超过80%的筛查检测到的癌症被人工智能归类为阳性。缺乏放射科医生是乳房x线摄影筛查的一个挑战。我们的发现支持了其他研究，这些研究表明使用人工智能可以减少屏幕阅读工作量。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

European Radiology 医学-核医学

CiteScore

11.60

自引率

8.50%

发文量

874

审稿时长

2-4 weeks

期刊介绍： European Radiology (ER) continuously updates scientific knowledge in radiology by publication of strong original articles and state-of-the-art reviews written by leading radiologists. A well balanced combination of review articles, original papers, short communications from European radiological congresses and information on society matters makes ER an indispensable source for current information in this field. This is the Journal of the European Society of Radiology, and the official journal of a number of societies. From 2004-2008 supplements to European Radiology were published under its companion, European Radiology Supplements, ISSN 1613-3749.