Clinician Trust and Human Factors in AI-Enabled Clinical Decision Support in Acute Care: Mixed Methods Study.

Abstract

Background: AI has the potential to enhance clinical decision-making in high-acuity settings such as intensive care units (ICUs) and emergency departments (EDs). However, despite promising performance, many AI-driven clinical decision support systems (AI-CDSSs) face poor adoption due to issues of trust, workflow disruption, and alert fatigue. Understanding the human factors that shape clinician acceptance is critical to guide safe and effective implementation of AI-CDSS in acute care. Theoretical frameworks, including the Systems Engineering Initiative for Patient Safety (SEIPS) 2.0 model and the technology acceptance model (TAM), suggest that successful adoption requires addressing sociotechnical interactions among clinician trust, system design, organizational readiness, and task complexity, yet few empirical studies have applied these frameworks to AI-CDSSs in acute care settings.

Objective: This study aimed to evaluate emergency medicine and critical care clinicians' perceptions of AI-CDSSs and to identify key factors influencing adoption, including trust, design preferences, and workflow integration.

Methods: A SEIPS 2.0-informed mixed methods study evaluated ICU and ED clinicians from Emory Healthcare on perceptions of AI in clinical practice. An expert-reviewed survey (N=57) assessed clinician perceptions, trust, and implementation preferences. Semistructured interviews (n=11) included A/B testing of AI-CDSSs and clinical sepsis scenarios to explore decision-making in context. Transcripts were thematically analyzed using the Braun and Clarke framework in ATLAS.ti (version 26, ATLAS.ti Scientific Software Development). Quantitative data were analyzed descriptively. This study assessed clinician perceptions using mock alerts and hypothetical scenarios rather than real-world AI-CDSS deployment.

Results: Trust in AI varied significantly by patient acuity (Cochran Q=30.40, P<.001): stable patients (43/57, 75%, 95% CI 63%-85%), deteriorating patients (27/57, 47%, 95% CI 35%-60%), and critically ill patients in the ICU and undifferentiated patients in the ED (25/57, 44%, 95% CI 32%-57% for each scenario). Internal consistency was acceptable-to-good across three scales (Cronbach α: AI Perception=.891, Trust=.743, Implementation=.740; McDonald ω: AI Perception=0.895, Trust=0.782, Implementation=0.746). Barriers included overreliance, insufficient training, and data quality concerns. For the exploratory AI-CDSS design, clinicians preferred opt-in alerts (10/11, 91%), evidence-linked recommendations (7/11, 64%), and avoiding overt mention of increased AI acceptance (8/11, 73%). Thematic analysis yielded 36 themes across six domains: trust and transparency, alert usability, workflow fit, data concerns, training needs, and perceived clinical impact. Clinicians favored AI-CDSSs that preserved autonomy, minimized disruption, and provided transparent rationales.

Conclusions: Adoption of AI-CDSSs in critical care is not solely a technical issue but a human-factors challenge centered on trust, transparency, and workflow compatibility. These findings support future testing of a phased implementation approach-beginning with lower-acuity applications where clinician trust is highest, then gradually extending to higher-acuity scenarios with enhanced transparency and override mechanisms. This graduated strategy addresses the critical interdependencies among people (trust), tools (design), organizations (training), and tasks (clinical complexity) identified in this study.