Collaborative Learning Environment over Internet Using Augmented Reality and Voice Interaction

Abstract

In this paper, we describe how we combined augmented reality based on marker tracking, 3D natural interaction using IBM Watson speech recognition capabilities on the cloud, and collaborative work allowing multiple users (students, teachers) to visualize in real-time and over the internet a same shared space mixing real scene with 3D contents of a biology course. Teachers and learners are able in such a way to interact collaboratively with the virtual objects in an intuitive and a natural manner by using voice commands and/or vocal inquiries to participate in the course and acquire knowledge. Each user is able to perform, during a real-time educational session, their own actions (selection, rotation, scale) over virtual objects and see and/or hear else’s actions and/or effects synchronously. Visual clues are added to enhance the context awareness and allow each user to recognize who else is acting/speaking at a given time in the group.