Stable Learning via Dual Feature Learning

Abstract

Stable learning aims to leverage the knowledge in a relevant source domain to learn a prediction model that can generalize well to target domains. Recent advances in stable learning mainly proceed by eliminating spurious correlations between irrelevant features and labels through sample reweighting or causal feature selection. However, most existing stable learning methods either only weaken partial spurious correlations or discard part of true causal relationships, resulting in generalization performance degradation. To tackle these issues, we propose the Dual Feature Learning (DFL) algorithm for stable learning, which consists of two phases. Phase 1 first learns a set of sample weights to balance the distribution of treated and control groups corresponding to each feature, and then uses the learned sample weights to assist feature selection to identify part of irrelevant features for completely isolating spurious correlations between these irrelevant features and labels. Phase 2 first learns two groups of sample weights again using the subdataset after feature selection, and then obtains high-quality feature representations by integrating a weighted cross-entropy model and an autoencoder model to further get rid of spurious correlations. Using synthetic and four real-world datasets, the experiments have verified the effectiveness of DFL, in comparison with eleven state-of-the-art methods.