From design to user experience: The creation and assessment of ASANA, an immersive VR task for situation awareness and spatial ability

Abstract

Accurate assessment of situation awareness (SA) and spatial ability (SpA) is critical in aviation, yet SA tools often interrupt tasks or offer limited temporal resolution, while SpA measures frequently rely on static 2D stimuli with low ecological validity. These limitations call for approaches that capture both abilities in realistic, dynamic contexts. Virtual reality (VR) offers this capability by enabling immersive 3D navigation while simultaneously recording performance and multimodal data. However, VR-based assessments must consider user-experience factors such as workload, affect, and simulator sickness, which can influence performance and the interpretability of assessment outcomes. Building on the preliminary study presented at IEEE VR’s Workshop on the eXtended Reality for Industrial and Occupational Supports, this paper describes the design of an immersive flight-navigation task that assesses SA and SpA, called Assessing Spatial Abilities in Naval Aviation (ASANA). We report user-experience outcomes from 106 U.S. Navy students, showing moderate workload, positive valence, near-neutral arousal, and slightly positive dominance. Simulator sickness increased from pre- to post-exposure, but post-exposure medians remained low, indicating generally mild symptoms. The correlation results showed that ASANA navigation efficiency aligned with established desktop-based SpA metrics. In addition, higher freeze-probe SA accuracy was associated with more efficient performance on an embedded, SME-informed in-scenario SA metric. Together, these findings support ASANA as a tolerable, interpretable VR platform for studying SpA and SA in ecologically-grounded context, and motivate future work that leverages synchronized multimodal sensing to model SA dynamics and SpA–SA interactions.