Potential Impact of an Artificial Intelligence-based Mammography Triage Algorithm on Performance and Workload in a Population-based Screening Sample.

Abstract

Objective: To evaluate potential screening mammography performance and workload impact using a commercial artificial intelligence (AI)-based triage device in a population-based screening sample.

Methods: In this retrospective study, a sample of 2129 women who underwent screening mammograms were evaluated. The performance of a commercial AI-based triage device was compared with radiologists' reports, actual outcomes, and national benchmarks using commonly used mammography metrics. Up to 5 years of follow-up examination results were evaluated in cases to establish benignity. The algorithm sorted cases into groups of "suspicious" and "low suspicion." A theoretical workload reduction was calculated by subtracting cases triaged as "low suspicion" from the sample.

Results: At the default 93% sensitivity setting, there was significant improvement (P <.05) in the following triage simulation mean performance measures compared with actual outcome: 45.5% improvement in recall rate (13.4% to 7.3%; 95% CI, 6.2-8.3), 119% improvement in positive predictive value (PPV) 1 (5.3% to 11.6%; 95% CI, 9.96-13.4), 28.5% improvement in PPV2 (24.6% to 31.6%; 95% CI, 24.8-39.1), 20% improvement in sensitivity (83.3% to 100%; 95% CI, 100-100), and 7.2% improvement in specificity (87.2% to 93.5%; 95% CI, 92.4-94.5). A theoretical 62.5% workload reduction was possible. At the ultrahigh 99% sensitivity setting, a theoretical 27% workload reduction was possible. No cancers were missed by the algorithm at either sensitivity.

Conclusion: Artificial intelligence-based triage in this simulation demonstrated potential for significant improvement in mammography performance and predicted substantial theoretical workload reduction without any missed cancers.