Detecting Speaking Intention From Motion and Physiological Data to Support Leadership in VR Group Discussions

Abstract

Effective leadership in virtual reality (VR) group discussions is challenged by the scarcity of nonverbal cues, making it difficult to infer members’ communicative readiness and manage turn-taking. This study introduces a multisensing approach to detect latent speaking intentions—both achieved and unachieved-by integrating motion data from commercial VR headsets with physiological signals. We propose a late-fusion deep learning model, omni-scale multivariate long short-term memory (OS-MLSTM) with cross attention, designed to capture temporal dependencies across modalities. Extensive validation shows that fusing motion and physiological features outperforms single-modality sensing in the mixed-subject setting with area under the receiver operating characteristic curve (AUROC) of 0.9057. Stricter validation settings, including leave-one-participant-out (AUROC: 0.7547) and purged cross validation (AUROC: 0.7752), indicate that unseen-user generalization still remains challenging. These findings provide a sensing foundation for leader-support mechanisms, such as real-time intention cues and summary dashboards, that may support leaders to identify latent opportunities for participation, thereby addressing problematic dynamics, such as intention bursts and participation imbalance, in VR group discussions.