Graph Transformer for Physical Rehabilitation Evaluation

Abstract

Physical rehabilitation is a medical specialty that focuses on the restoration of body functions in the safest and most effective way possible. During a rehabilitation exercise session, patients' behavior reflects their health status and is an important indicator of the treatment outcome. Building an automatic system for the evaluation of human motion quality in an objective and reliable way can be used in medicine to establish a differential diagnosis, to choose the adequate treatment, or for patient monitoring. Deep learning has become the state-of-the-art in Human Action and Human Behavior Recognition from videos. Most of the state-of-the-art model architectures are CNNs based and often use RNNs to calculate temporal dependencies among frames, and ignore the topological structure of the human body. In this work, we propose to use a Graph Transformer network with spatial and temporal attention mechanisms for physical rehabilitation evaluation. First, we used a standard Transformer network with a selfattention mechanism, then we take advantage of graph skeletons as inputs for a two-stream spatio-temporal graph network with both spatial and temporal attention mechanisms. We used our model on UI-PRMD, a benchmark dataset that provides skeleton data using motion capture systems. Our results show that our attention-based based ST-GCN models outperform the state-of-the-art methods on quality score prediction and binary classification.