A Digital Reality Theater with the Mechanisms of Real-Time Spoken Language Evaluation and Interactive Switching of Scenario & Virtual Costumes: Effects on Motivation and Learning Performance

To promote language learning performance, a realistic speech mission-based digital learning theater, which integrates with real-time spoken language evaluation and interactive switching of scenario and virtual costumes, was proposed in this study. Regarding evaluation, an AI chatbot is integrated into the digital learning theater to assign speech tasks to learners and evaluate their performance in terms of pronunciation and grammar, and then provide immediate feedback for improvement. Regarding interaction, with Kinect motion detection technology, learners can immerse themselves into the digital scene of drama with their whole body and wear various virtual costumes that can automatically match learners’ body size and motions, to strengthen learners’ interest and engagement in learning. This study was conducted on a Japanese language course for university students. The results suggest that students’ learning outcomes and motivation can be enhanced with realistic speech mission-based digital learning theater.