Driver’s facial expression recognition by using deep local and global features

Abstract

Understanding drivers’ emotions is crucial for safety and comfort in autonomous vehicles. While Facial Expression Recognition (FER) systems perform well in controlled environments, struggle in real driving situations. To address this challenge, an Interlaced Local Attention Block within a Convolutional Neural Network (ILAB-CNN) model has been proposed to analyze drivers’ emotions. In real-world scenarios, not all facial regions contribute equally to expressing emotions; specific areas or combinations are key. Inspired by the attention mechanism, an ILAB and a Modified Squeeze-and-Excitation (MSE) block has been proposed to learn more discriminative features. The MSE block applies a self-attention mechanism on the channels, effectively identifying key features by incorporating global information and discarding irrelevant features. ILAB employs the MSE and encoder-decoder structures for region-channel specific attention in one branch and combines it with the obtained feature map of the MSE from the other branch. The proposed approach successfully captures essential information from facial expressions while utilizing a reduced number of parameters, leading to significantly improved recognition accuracy and recognition time for real-time applications. Evaluated on diverse datasets, our method shows 75.3 % recognition rate on FER-2013, 85.06 % on RAF-DB, and 98.8 % on KMU-FED, demonstrating its potential to advance FER technology.