Adaptive Training Environment without Prior Knowledge: Modeling Feedback Selection as a Multi-armed Bandit Problem

Abstract

Pedagogical Action Selection (PAS) is a major issue for intelligent tutoring and training systems. Expert knowledge provides useful insights to build strategies that relate students representation to PAS, but it can be difficult to collect. Furthermore, the influence of a specific action may vary across students, which is rarely reflected in expert knowledge. As part of an automatic gesture training system, we propose to model the co-evolution between a student and a training environment in order to provide personalized action selection. The proposed approach is based on three models representing the student, the environment, and the interactions between these two entities. The latter model sees the PAS as a multi-armed bandit problem, each arm representing a possible action. Thus, PAS personalization only relies on the interactions between the student and the learning environment, without any prior knowledge. Two experiments, one in a simulated environment and a second in a calligraphy training environment, highlight the model ability to personalize action selection, and the benefits of this ability on students skill acquisition.