Quantifying Healthcare Provider Perceptions of a Novel Deep Learning Algorithm to Predict Sepsis: Electronic Survey.

Abstract

Importance: Sepsis is a major cause of morbidity and mortality, with early intervention shown to improve outcomes. Predictive modeling and artificial intelligence (AI) can aid in early sepsis recognition, but there remains a gap between algorithm development and clinical use. Despite the importance of user experience in adopting clinical predictive models, few studies have focused on provider acceptance and feedback.

Objectives: To evaluate healthcare worker perception and acceptance of a deep learning sepsis prediction model in the emergency department (ED).

Design, setting, and participants: COnformal Multidimensional Prediction Of SEpsis Risk (COMPOSER), a deep learning algorithm, is used at two EDs of a large academic medical center to predict sepsis before clear clinical presentation. An internally developed survey following the Checklist for Reporting Results of Internet E-Surveys was distributed to team members who received a COMPOSER alert.

Analysis: Mann-Whitney U testing was performed on results stratified by provider experience.

Results: A total of 114 responses were received: 76 from doctors of medicine/doctors of osteopathic medicine, 34 from registered nurses, and four from nurse practicioners/physician assistants. Of these, 53% were from providers with fewer than 5 years of experience. Seventy-seven percent of respondents had a positive or neutral perception of the alert's usefulness. Providers with 0-5 years of experience were more likely to expect sepsis after the alert (p = 0.021) and found the alert more useful (p = 0.016) compared with those with 6+ years of experience. Additionally, physicians with 0-5 years of experience were more likely to say the alert changed their patient management (p = 0.048).

Conclusions: Less experienced providers were more likely to perceive benefit from the alert, which was overall received favorably. Future AI implementations might consider tailored alert patterns and education to enhance reception and reduce fatigue.