Metalogue: A Multimodal Learning Journey

Abstract

In this paper, we present a high-level description of the Metalogue system that develops a multi-modal dialogue system that is able to implement interactive behavior between a virtual agent and a learner outlining the insight to the development of a fully-integrated multimodal interactive system. This system includes several components addressing several research domains: metacognitive modeling, skill training, usability testing, prosody analysis, multimodality, dialogue management, speech recognition, gesture recognition and interpretation, and learner feedback. The key issue is the integration of all these components in a single platform, allowing to the users to improve their metacognitive skills. This work reports on the user experience evaluation during the design and development phase of the system that feed back to the design and continuous refinement of the overall approach.