Real-time Fitness Activity Recognition and Correction using Deep Neural Networks

Abstract

Fitness activities are beneficial to one's health and well-being. During the Covid-19 pandemic, demand for virtual trainers increased. There are current systems that can classify different exercises, and there are other systems that provide feedback on a specific exercise. We propose a system that can simultaneously recognize a pose as well as provide real-time corrective feedback on the performed exercise with the least latency between recognition and correction. In all computer vision techniques implemented so far, occlusion and a lack of labeled data are the most significant problems in correctly detecting and providing helpful feedback. Vector geometry is employed to calculate the angles between key points detected on the body to provide the user with corrective feedback and count the repetitions of each exercise. Three different architectures-GAN, Conv-LSTM, and LSTM-RNN are experimented with, for exercise recognition. A custom dataset of Jumping Jacks, Squats, and Lunges is used to train the models. GAN achieved a 92% testing accuracy but struggled in real-time performance. The LSTM-RNN architecture yielded a 95% testing accuracy and ConvLSTM obtained an accuracy of 97% on real-time sequences.