Motion data alignment and real-time guidance in cloud-based virtual training system

Abstract

In this paper, by making use of virtual reality technology, motion sensors and cloud computing platform, we propose a cloud-based virtual training system for physical therapy, which enables a user to be trained by following a pre-recorded avatar instructor and getting real-time guidance using mobile device through wireless network. To evaluate the user's performance, we compare the motion data of the user and the pre-recoded avatar instructor. However, human reaction delay and network delay cause the data misalignment problem in the proposed cloud-based virtual training system. To align the motion data and evaluate the user's performance, we use Dynamic Time Warping (DTW) to calculate the similarity between the two sequences. Moreover, we propose a variant of the DTW algorithm we term Gesture-Based Dynamic Time Warping (GB-DTW) which segments the whole motion sequence and provides evaluation score for each gesture in real time. Experiments with multiple subjects under real network condition show that the proposed GB-DTW algorithm performs much better than other evaluation methods. To help the user calibrate his movements, the proposed system also provides visual and textual guidance for the user.