A prototype of a self-motion training system based on deep convolutional neural network and multiple FAMirror

Abstract

With the development of deep learning methods, there has been a significant development in motion and speech recognition technologies, which have become common methods in Human-Computer Interaction (HCI). In addition, a mirror-metaphor is something that can be easily found around us, and it has become one of the displays for augmented reality as it enables participants to observe themselves. This paper proposes a prototype of self-motion training AR system based on these two important aspects. In the self-motion training system, we propose a method to represent one motion as one image. This method enables faster deep learning and motion recognition. For a self-motion training system, there are two essential requirements. One is that the participants should have the ability to observe their motion as well as a reference motion model, and it should be possible to correct their motion by comparing with the reference model. The other requirement is that the system could recognize a participant's motion from among various motion models in a database. Here, we introduce the configuration of a self-motion training system based on AR and its implementation details. In addition, the system examines the accuracy of the participant's motion with a reference motion model.