Facial expression-based emotion recognition across diverse age groups: a multi-scale vision transformer with contrastive learning approach

Abstract

Facial expression-based Emotion Recognition (FER) is crucial in human–computer interaction and affective computing, particularly when addressing diverse age groups. This paper introduces the Multi-Scale Vision Transformer with Contrastive Learning (MViT-CnG), an age-adaptive FER approach designed to enhance the accuracy and interpretability of emotion recognition models across different classes. The MViT-CnG model leverages vision transformers and contrastive learning to capture intricate facial features, ensuring robust performance despite diverse and dynamic facial features. By utilizing contrastive learning, the model's interpretability is significantly enhanced, which is vital for building trust in automated systems and facilitating human–machine collaboration. Additionally, this approach enriches the model's capacity to discern shared and distinct features within facial expressions, improving its ability to generalize across different age groups. Evaluations using the FER-2013 and CK + datasets highlight the model's broad generalization capabilities, with FER-2013 covering a wide range of emotions across diverse age groups and CK + focusing on posed expressions in controlled environments. The MViT-CnG model adapts effectively to both datasets, showcasing its versatility and reliability across distinct data characteristics. Performance results demonstrated that the MViT-CnG model achieved superior accuracy across all emotion recognition labels on the FER-2013 dataset with a 99.6% accuracy rate, and 99.5% on the CK + dataset, indicating significant improvements in recognizing subtle facial expressions. Comprehensive evaluations revealed that the model's precision, recall, and F1-score are consistently higher than those of existing models, confirming its robustness and reliability in facial emotion recognition tasks.