An Ensemble Model using Face and Pose Tracking for Engagement Detection in Game-based Rehabilitation

Abstract

Highly engaging rehabilitation promotes functional reorganization of the brain in stroke patients. Engagement detection in game-based rehabilitation can help rehabilitation practitioners get real-time feedback, and then provide patients with appropriate training programs. Previous research on engagement detection has focused on wearable devices, and the complicated laboratory setup makes them unsuitable for use in clinics and homes. In this work, we propose a method to automatically extract facial and posture features from camera-captured videos. Then we design an automatic engagement detection model using the facial and posture features as the input. In the dataset of engagement in virtual game rehabilitation scenarios, our model detects engagement levels with an average accuracy of 96.85%, achieving remarkable performance. This study sheds new light on engagement detection for stroke patients in clinical applications.