Automatic recognition and assessment of physical exercises from RGB images

Abstract

Physical exercises are important for a healthy life. However, many people do the exercises without professional assistance, especially when practicing at home during Covid-19. Inappropriate exercising can negatively impact and even result in muscle pain. In this paper, an exercise coaching application is developed to understand what the user is doing and provide useful assessments and guidelines to assist the users. The proposed application takes RGB image sequences from any off-the-shelf cameras widely integrated into smartphones or laptops as input. First, skeleton sequences are extracted from RGB images using the public tool Google MediaPipe. Then, a real-time action recognition based on the temporal sliding window and DD-Net model is proposed to determine the action class. Two frame-based and sequence-based scores are estimated to provide a quantitative assessment. Finally, a tool with GUI and a database are developed.