Towards Immersive Skill Training for First Responders with Biosensor-based Assessment of Situation Awareness

Abstract

First responders engage highly stressful situations at the emergency site that may induce stress, fear, panic and a collapse of clear thinking (Putnam, 1995). However, their physiological and cognitive readiness is of highest importance to enable appropriate decision making (Frye & Wearing, 2014). A major aspect for keeping control of the situation is that first responders should always maintain situation awareness. Training of the first responders’ routines improves resilience towards stressors in severe hazard conditions. Advanced training of first responders, in particular, the use of VR environments for the training of situation awareness for stress-resilient decision-making behavior is one of the upcoming challenges of the near future.Methods A Virtual Reality environment was developed with a typical firefighter scenario to train and to evaluate the situation awareness (SA) of squad leaders. The operator has two major tasks, (i) searching for pocket of embers on the ground by using an infrared camera and whenever detected to call for a team member to work on it, (ii) to survey her team of 5 first responders for their visible presence as well as for their health condition and consequently to react in proper time instants whenever team members would be injured or disappear. The movement of the team members within a predefined skill-oriented territory is activated by an AI-driven method. A wearable multisensory-based non-invasive measurement suite is mounted on the operator including VR in-built eye-tracking, biosensors for cardiovascular, electrodermal and temperature data capture, to report by means of digital human factors analytics. From the sensor data stream we intend to deduce psychological constructs, such as, situation awareness, physiological strain, cognitive-emotional stress, and fatigue. A pilot study is planned with N=20 students, with pre- and post-study cognitive tests, such as, determination test (Schuhfried, 1987) and Psychomotor Vigilance Test (PVT; Dinges & Powell, 1985), as well as the Situation Awareness Rating Technique (SART; Taylor, 1990). From the SART we will research for to situation awareness-specific digital biomarkers by means of correlation analysis. This multi-tasking configuration will enable to measure (a) executive functions, such as, cognitive flexibility, and (b) level 1, 2 and 3 SA by gaze-driven virtual events, such as, viewing first responder avatars. Results: We will describe the system architecture as well as the features of the VR-based skill evaluation system (VR with eye-tracking, biosensors, treadmill, dashboard). Test users will move on a treadmill that triggers the VR-based experience accordingly, by means of a Cyberith Visualizer platform. Videos that were captured from the use of the platform and application of biosensors with eye tracking will demonstrate the usefulness for the evaluation of SA. We will report from a first usability study with feedback from friendly users. At the conference, we will be able to report on first results of the pilot study.Conclusion: Advanced training of first responders and emergency staff with typical operation scenarios is one of the upcoming challenges of the near future. First results motivate the use of VR environments for the training of situation aware and stress-resilient decision-making behavior of firefighters.