Classification of physical exercise intensity by using facial expression analysis

Facial expression analysis has a wide area of applications including health, psychology, sports etc. In this study, we explored different methods of automatic classification of exercise intensities using facial image processing of a subject performing exercise on a cycloergometer during an incremental standardized protocol. The method can be implemented in real time using facial video analysis. The experiments were done with images extracted from a 12 min HD video collected in laboratorial normalized settings (TechSport from the University of Trás-os-Montes e Alto Douro) with a static camera (90° angle with face and camera). The time slot for video to extract images for a particular class of exercise intensity is correspondence to the incremental heart rate. The facial expression recognition has been performed mainly in two steps: facial landmark detection and classification using the facial landmarks. Luxand application was used to detect 70 landmarks were detect using the adaptation of code available in Luxand application and we applied machine learning classification algorithms including discriminant analysis, KNN and SVM to classify the exercise intensities from the facial images. KNN algorithms presents up to 100% accuracy in classification into 2 and 3 classes. The distances between a lowermost landmark of the faces, which is indicated in landmark number 11 in the Luxand application, and the 26 landmarks around mouth were calculated and considered as features vector to train and test the classifier. Separate experiments were done for classification into two, three, and four classes and the accuracy of each algorithm was analyzed. From the overall results, classification into two and three classes was easy and resulted in very good classification performance whereas the classification with four classes had poor classification performance in each algorithm. Preliminary results suggest that distinguishing more levels of exertion, might require additional feature variables.