Using augmented reality filters to display time-based visual cues

Abstract

Introduction: Healthcare education commonly uses practices like moulage to represent visual cues (e.g., symptoms). Unfortunately, current practices have limitations in accurately representing visual symptoms that develop over time. To address this challenge, we applied augmented reality (AR) filters to images displayed on computer screens to enable real-time interactive visualizations of symptom development. Additionally, this study explores the impact of object and filter fidelity on users’ perceptions of visual cues during training, providing evidence-based recommendations on the effective use of filters in healthcare education. Methods: We conducted a 2 × 2 within-subjects study that involved second-year nursing students (N = 55) from the University of Florida. The study manipulated two factors: filter fidelity and object fidelity. Filter fidelity was manipulated by applying either a filter based on a medical illustration image or a filter based on a real symptom image. Object fidelity was manipulated by overlaying the filter on either a medical manikin image or a real person image. To ensure that potential confounding variables such as lighting or 3D tracking did not affect the results, 101 images were pre-generated for each of the four conditions. These images mapped to the transparency levels of the filters, which ranged from 0 to 100. Participants interacted with the images on a computer screen using visual analog scales, manipulating the transparency of the symptoms until they identified changes occurring on the image and distinct symptom patterns. Participants also rated the severity and realism of each condition and provided feedback on how the filter and object fidelities impacted their perceptions. Results: We found evidence that object and filter fidelity impacted user perceptions of symptom realism and severity and even affected users’ abilities to identify the symptoms. This includes symptoms being seen as more realistic when overlaid on the real person, symptoms being identified at earlier stages of development when overlaid on the manikin, and symptoms being seen as most severe when the real-image filter was overlayed on the manikin. Conclusion: This work implemented a novel approach that uses AR filters to display visual cues that develop over time. Additionally, this work’s investigation into fidelity allows us to provide evidence-based recommendations on how and when AR filters can be effectively used in healthcare education.