Remote practical instruction using web browsers

Abstract

This paper introduces a novel approach to remote coaching, specifically targeting the body movements of learners participating remotely. The proposed system employs a smartphone camera to capture the learner’s body and represent it as a 3D avatar. The instructor can then offer guidance and instruction by manipulating the 3D avatar’s shape, which is displayed on a web browser. The main challenge faced by the system is to enable the sharing and editing of 3D objects among users. Since the HTML5 drag-and-drop feature is inadequate for transforming virtual objects consisting of multiple interconnected rigid bodies, the system tracks the pivot point of the manipulated rigid body. It assigns attributes such as pivot points and action points to each object, extending beyond their 2D screen coordinates. To implement the system, an interactive web application framework following the model-view-view-model (MVVM) architecture is utilized, incorporating Vue.js, Three.js, and Google Firebase. The prototype system takes advantage of the data binding capability of the framework and successfully operates within the 3D space of a web browser. Experimental results demonstrate that it can effectively share transformation information with an average delay of 300 ms.