Stable Learning via Triplex Learning

Abstract

Stable learning aims to learn a model that generalizes well to arbitrary unseen target domain by leveraging a single source domain. Recent advances in stable learning have focused on balancing the distribution of confounders for each feature to eliminate spurious correlations. However, previous studies treat all features equally without considering the difficulties of confounder balancing associated with different features, and regard irrelevant features as confounders, deteriorating generalization performance. To tackle these issues, this article proposes a novel triplex learning (TriL) based stable learning algorithm, which performs sample reweighting, causal feature selection, and representation learning to remove spurious correlations. Specifically, first, TriL adaptively assigns weights to the confounder balancing term of each feature in accordance with the difficulties of confounder balancing, and aligns the confounder distribution of each feature by learning a group of sample weights. Second, TriL integrates the sample weights into a weighted cross-entropy model to compute causal effects of features for excluding irrelevant features from the confounder set. Finally, TriL relearns a set of sample weights and uses them to guide a new supervised dual-autoencoder containing two classifiers to learn feature representations. TriL forces the results of two classifiers to remain consistent for removing spurious correlations by using a cross-classifier consistency regularization. Extensive experiments on synthetic and two real-world datasets show the superiority of TriL compared with seven methods.