A novel facial expression recognition method based on cross direction attention network

Facial expression recognition (FER) is an area of growing interest in computer vision research. This paper extends the framework provided by the ‘Distract your Attention Network’ (DAN) which consists of multiple parallel branches, each branch composes of a spatial attention (SA) module followed by a channel attention (CA) module, and then these multiple branches are fused together before being passed into a classifier module. The spatial attention module of DAN has an internal channel dimension of 1, while our proposed Cross Directional Attention Network (CDAN)-I and CDAN-II contain respectively an internal channel dimension of 512 (same as the channel dimension of the input), and internal channel dimension of 1024 (double that of the channel dimension of the input). These increases in internal channel dimension allow extraction of more features, before they are being made to conform with the input channel dimension. Despite these seemingly simple modifications from that of DAN, both CDAN-I and CDAN-II are found to outperform those of DAN, a state-of-the-art FER method, on four popular FER benchmark datasets: RAF-DB (Real world Affective Face-database), AffectNet-7 (AffectNet with Seven Categories) AffectNet-8 ( AffectNet with Eight Categories), and CK+ (Cohn–Kanada Extended). Moreover, we make use of three statistical indexes for clustering analysis, and verified that the CDAN-I and CDAN-II modules have been able to increase the inter-cluster distances, and decrease the intra-cluster distances, when compared with those obtained by the backbone ResNet-18 network (Residual Network with 18 Layers) , thus providing a quantitative analysis technique in this area.