SELFIE-Net: Self-Supervised Learning and Feature-Intensified Emotion Network for Facial Emotion Recognition

Abstract

As the requirement for the strong facial emotion recognition (FER) systems in different applications increases, both in healthcare and security, it is important to address issues such as multiple facial expressions and complicated backgrounds. This article proposes a new FER framework called SELFIE-Net that uses “self-supervised learning” combined with “diffusion-based classification head” on both labelled and unlabelled data. The model combines convolutional block attention module (CBAM) to refine the extraction of features. The classification and regression diffusion model (CARD) enhances the prediction of emotional conditions. SELFIE-Net is a modularly integrated system with the power of SSL, and has been applied to bootstrap your own latent (BYOL), Barlow twins, DINO (self-distillation with no labels), and SimCLR to learn the significant features of the unlabeled data. Comprehensive tests on benchmarking datasets such as the FER2013, RAF-DB, and AffectNet show that SELFIE-Net achieves superior performance in comparison to current approaches, establishing new standards in the accuracy and flexibility of FER. This study is not only a contribution to the current body of emotion recognition, but also it shows the possibility of integrating self-supervised learning and attention mechanisms to tackle more complex problems of pattern recognition.