Mixed reality alters motor planning and control

Abstract

Compared to physical unmediated reality (UR), mixed reality technologies, such as Virtual (VR) and Augmented (AR) Reality, entail perturbations across multiple sensory modalities (visual, haptic, etc.) that could alter how actors move within the different environments. Because of the mediated nature, goal-directed movements in VR and AR may rely on planning and control processes that are different from movements in UR, resulting in less efficient motor control. The current study involved participants performing manual pointing movements on Müller-Lyer illusion stimuli to examine the relative contributions of movement planning and online control in UR, VR, and AR. Compared to UR, movements in VR were slower but were equally variable with a comparable level of online control, whereas movements in AR showed comparable speed but exhibited higher variability and less online control. Further, movements in VR and AR demonstrated a greater illusory effect in endpoint accuracy relative to UR. These findings suggested that participants in VR adopted an active compensation strategy to overcome the impact of less efficient online control, whereas participants in AR did not. The findings that movement planning and execution in VR and AR are fundamentally different from those in UR provide valuable insights into the potential neural systems engaged during movements in different realities.