Hybrid Learning Based Visual Facial Emotion Recognition in Speech Videos

Abstract

Facial Emotion Recognition (FER) plays a crucial role in understanding human behavior and interactions, with applications in human-computer interaction, healthcare, surveillance, and multimedia content analysis. Emotion recognition from speech videos is particularly challenging as it requires capturing and analyzing subtle variations in facial expressions along with dynamic speech changes. In this paper, a hybrid learning approach is presented that integrates machine learning and deep learning models to enhance the accuracy and robustness of visual facial emotion recognition in speech videos. The proposed approach leverages a pre-trained ResNet-101 model for feature extraction and evaluates three widely used machine learning classifiers, Random Decision Forest (RDF), Decision Tree Boosting (DTB), and Support Vector Machine (SVM) for emotion prediction. Experiments conducted on the RAVDESS dataset demonstrate that our hybrid model, combining Deep CNN with RDF classification, achieves a high accuracy of 92.50%, outperforming existing state-of-the-art models. This work highlights the potential of integrating deep learning and machine learning for FER, offering significant benefits for applications in healthcare, education, and human-computer interaction.