Special Issue on CDS Failures: Development and Evaluation of ORCA, a Resilient Solution for Order Set Access During EHR Downtimes.

Abstract

Background: Clinical decision support systems (CDSS) are central to modern healthcare, but their effectiveness is compromised during system downtimes, which affect 96% of healthcare organizations. During these failures, clinicians lose access to critical decision-making tools like order sets, increasing the risk of medical errors. Traditional downtime solutions, such as paper-based protocols, are often impractical and difficult to maintain.

Objectives: This study introduces and evaluates ORCA (Offsite Repository for Clinical Assets), a resilient web-based solution designed to maintain access to EHR order sets during system failures. We assessed its usability and effectiveness as a downtime decision support tool across various clinical settings.

Methods: ORCA was developed based on analysis of previous downtime incidents, replicating essential order set functionality while ensuring offsite accessibility. We conducted usability testing with 16 clinicians from diverse specialties, using structured tasks and think-aloud protocols. User feedback was collected through the Usability Metric for User Experience (UMUX) questionnaire and thematic analysis of interview transcripts.

Results: ORCA demonstrated strong usability (mean UMUX score: 86.2). Thematic analysis revealed key implementation challenges: system limitations (24.56% of responses), workflow integration (23.39%), and interface navigation (22.22%). Users valued ORCA's familiar interface and offsite accessibility but identified critical gaps in dynamic decision support capabilities.

Conclusions: ORCA represents a viable approach to maintaining basic clinical decision support during downtimes. However, significant challenges remain in replicating dynamic CDS features and ensuring effective integration with existing downtime procedures. These findings inform future development of resilient CDS systems and highlight the importance of planned fallback pathways in clinical systems.