Development and Evaluation of a Multimodal Debriefing Dashboard for Virtual Reality Cardiac Arrest Simulation.

Abstract

Introduction: Wearable sensing technologies incorporated into virtual reality (VR) headsets can provide a number of unique, previously hidden insights into individual and team learning and performance in complex environments. However, the most impactful and efficient mechanisms for conveying multimodal data (e.g., gaze, mental workload, etc) allowing for unique insights are not well understood. This study used a human-centered design process to develop and evaluate a multimodal debriefing dashboard.

Methods: Twelve Advanced Cardiac Life Support (ACLS) Instructors completed VR cardiac arrest simulations and participated in interviews and design thinking workshops to identify dashboard features, leading to high-fidelity mock-ups and an interactive prototype. A second group of 6 ACLS instructors then evaluated the prototype by providing mock feedback on simulation performance with and without the dashboard in a pre-post format, followed by surveys assessing usability, acceptability, appropriateness, feasibility, trust, and perceived accuracy.

Results: The final prototype dashboard features a video, a timeline of clinical actions, automatic error identification, mental workload, and visual attention data. It also generates closed-loop communication reports on a separate webpage. The evaluation study showed that participants provided performance analysis and feedback to students when using the dashboard, targeting specific (non)technical skills aligned with ACLS learning objectives. Participants rated the dashboard highly, with a System Usability Scale score of 88.9%, reflecting above-average usability.

Conclusions: An interactive multimodal debriefing dashboard prototype was developed using a user-design framework with prototyping and iterative feedback. Use of the dashboard resulted in improved evaluation and debriefing practices targeting learning outcomes, with participants rating the dashboard favorably. This dashboard has the potential to enhance simulation-based learning by offering near real-time analytics that promote a deeper understanding of individual and team technical and nontechnical skill acquisition and mastery.