Unsupervised Cross-Domain Facial Expression Recognition via Class Adaptive Self-Training

Abstract

Unsupervised Cross-Domain Facial Expression Recognition (CD-FER) aims to transfer the recognition ability from annotated source domains to unlabeled target domains. Despite the advancements in CD-FER techniques based on marginal distribution matching, certain inherent properties of facial expressions, such as implicit class margins and imbalanced class distributions, still leave room for improvement in existing models. In this paper, we propose a Class-Adaptive Self-Training (CAST) model for unsupervised CD-FER. In addition to domain alignment, the CAST model leverages self-training to learn pseudo labels and dually enhance aligned representations for explicit class distinction, considering implicit class margins. Furthermore, the CAST model conducts a comprehensive analysis of the negative effects of class distributions on pseudo-label learning from perspectives of class-level representation distributions and predicted probabilities, and subsequently proposes specific solutions. By jointly matching class-level representation distributions and class distributions, the CAST model successfully alleviates conditional distribution discrepancies between domains, which is particularly pertinent for facial expression properties. Experimental results, including assessments on multiple target domains and evaluations of multiple FER models, demonstrate the effectiveness, superiority, and universality of the CAST model.