Exploring user reception of speech-controlled virtual reality environment for voice and public speaking training

Abstract

In this paper, we explore the development and assessment of a virtual reality (VR) system designed to enhance public speaking and vocal skills among professional and non-professional speech users alike. The system’s foundation lies in a speech recordings corpus of 529 utterances given during presentations by a total of 15 students. From these data, we extracted voice parameters such as pitch, timbre, and speech rate using speech processing methods. We also asked six expert annotators to evaluate the stress levels present within each presentation. This multi-faceted analysis facilitated the selection of specific parameters for real-time animation control of virtual characters responding dynamically to the change in the speaker’s voice. Through these mechanics, we could cultivate user proficiency in voice modulation, thereby improving overall speaking abilities and confidence. Furthermore, the system fosters self-awareness of vocal quality, promoting proper utilization of the voice in professional settings. Our VR system offers a dual-mode environment that combines traditional public speaking scenarios in front of a virtual audience with a relaxing forest setting, where users can control weather conditions with their voice. To assess the system’s efficacy, we conducted a pilot study with five participants. Additionally, we provide preliminary design guidelines informed by our user study to support the development of future VR-based speech trainers.