Exploring the impact of virtual reality flight simulations on EEG neural patterns and task performance

Abstract

Neurophysiological measurements, such as electroencephalography (EEG), can be used to derive insight into pilots’ mental states during flight training and to track learning progress in order to optimize the training experience for each individual. Prior work has demonstrated that the level of fidelity of a flight simulation (2D Desktop vs. 3D VR) is associated with different cortical activity in relation to task demands. However, it remains unknown whether simulation fidelity affects flight performance, and whether this effect can be observed in EEG neurophysiological responses associated with workload. The current study therefore assessed whether an EEG-based index of workload and task engagement is predictive of performance during flight training in different simulation environments. We conducted a within-subject designed experiment with 53 novice participants who performed two flight tasks (speed change, medium turn) under two conditions (Desktop vs. VR). EEG signals were collected throughout the experiment to quantify mental workload using the beta-ratio ($\frac{\beta }{\alpha +\theta }$). The VR condition showed increased beta-ratios in all lobes, including frontal and parietal areas, compared to the Desktop simulation. Additionally, we observed an effect of simulator environment on performance, as VR was associated with improved flight performance. However, we found no evidence of a relationship between the beta-ratio and performance. Our findings demonstrate that the brain responds differently to tasks in training environments of various levels of fidelity. However, more research into the neurometrics of flight training is needed in order to develop brain-computer interfaces that can enhance current pilot training methods by providing personalized feedback in real-time.