The Role of Time in Facial Dynamics and Challenges in Automatic Emotion Recognition (2019–2024)

Abstract

Based on a comprehensive literature review, this study highlights the critical role of the temporal dimension of facial dynamics in understanding facial expressions and improving the accuracy and robustness of automatic emotion recognition systems (machine-FER). While deep learning (DL) techniques like convolutional neural networks (CNNs) and long short-term memory (LSTM) networks offer significant advances, they face challenges such as gradient vanishing and overfitting, particularly in long and complex sequences. Vision transformers (ViTs) show promise but require integration with algorithms to mitigate spatial noise. Conventional machine learning (CML) methods like support vector machine (SVM) remain robust, especially in smaller datasets. The study underscores the importance of multimodal data synchronization (e.g., video, voice) in classifying emotions more accurately, reflecting both human and machine learning capabilities. It also addresses the limitations of current models, including cultural biases and the need for large, diverse datasets. The findings suggest that future research should focus on real-world conditions, integrating sequential multimodal data and employing supervised models based on theoretical and empirical frameworks. This approach is aimed at enhancing the understanding and classification of facial emotions, ensuring data quality and acceptable results through systematic human observations. The study provides valuable insights for selecting appropriate algorithms that are tailored to specific research objectives and contexts.