Efficient hierarchical CNN model with self-attention for three-category facial emotion tracking in healthcare applications

Abstract

Facial emotion recognition (FER) is a transformative tool for advancing healthcare, particularly in telehealth, where non-verbal communication is limited. This paper introduces Facial Emotion Tracking for Healthcare Applications (FET4H), a lightweight and effective FER system that leverages a hierarchical CNN architecture with integrated self-attention mechanisms to achieve high accuracy in recognizing key emotional states—negative, neutral, and positive. It also addresses challenges such as lighting variability, pose differences, and facial obstructions. Validated on real-world datasets, including RAF-DB, FER+, and AffectNet, FET4H demonstrates superior performance compared to existing methods. Integrated into user-friendly software, FET4H supports real-time emotion tracking, data logging, and telehealth integration, aiding providers in assessing satisfaction, detecting distress, and personalizing care. Its flexibility for retraining and customization ensures adaptability across scenarios, while its efficiency reduces operational costs for healthcare organizations. FET4H also serves as a benchmark for FER research, inspiring advancements in technology and interdisciplinary collaboration, ultimately enhancing telehealth outcomes and enabling equitable, high-quality remote healthcare.