Special Issue on CDS Failures: The Costs and Benefits of Clinical Decision Support for Radiology Appropriate Use Criteria: A Retrospective Observational Study.

Background: Appropriate Use Criteria Clinical Decision Support (AUC CDS) was legislatively mandated in the United States in 2014, and multiple CDS vendors were designated as qualified Clinical Decision Support Mechanisms by the Centers for Medicare and Medicaid Services. Little is known about the costs and benefits of these systems in real-world settings.

Objectives: We evaluated the effectiveness of an AUC CDS system and the time costs it imposes on clinicians at a US academic medical center.

Methods: Our academic medical center's enterprise data warehouse was queried for AUC CDS alert events and timestamps occurring between July 1, 2021 and June 30, 2022. We calculated percent of altered orders and alert-related timespans, and used these to calculate CDS positive predictive value (PPV), time costs, and the cost-benefit ratio of minutes of provider time per altered order. Based on the medical literature and expert opinion on well-performing CDS, we hypothesized a CDS PPV of 8%.

Results: Overall PPV was 1%, leading us to reject our hypothesis that our AUC CDS was well performing (p < 0.001). Median time costs per alert were high (12 seconds load time, 2 seconds dwell time), yielding a CDS cost/benefit ratio of 38 provider minutes per altered order.

Conclusions: Despite using one of three market-leading AUC CDS tools, our CDS demonstrated long load times, short dwell times, and low PPV. Provider attention is not free - policymakers should consider both CDS effectiveness and costs (including time costs) when designing AUC policy.