Robust semi-supervised learning with reciprocal weighted mixing distribution alignment

Recent semi-supervised learning(SSL) methods have achieved great success owing to the impressive performances brought by the combination of pseudo-labeling and consistency regularization. These methods often use pre-defined constant thresholds or dynamical thresholds to select unlabeled samples that contribute to training. However, many correct/incorrect pseudo-labels may be ignored/selected. Especially in distribution mismatched scenario, threshold-adjusted strategy is often complex and ineffective. To alleviate this issue, we develop a simple yet powerful framework whose idea is to abandon this strategy and utilize distribution alignment to adjust the predictions generated from a biased model softly. Specifically, first, we create two classifiers to predict pseudo-label(i.e., the sample belongs to a specific category) and complementary pseudo-label(i.e., the sample does not belong to a specific category), respectively. Second, by maintaining the distributions of pseudo-labels, complementary pseudo-labels and their reverse versions from past iterations, we enforce a reciprocal weighted mixing according to the predicted category weights. Third, a reciprocal distribution alignment is applied to the mixed distributions to adjust the predicted distributions. Finally, we propose Implication Alignment Loss , which keeps consistency between the predictions of the same implications but from different versions. We empirically demonstrate the effectiveness of our proposed method in comparison with state-of-the-art benchmarks. Especially, our method achieves a 1.18% error rate reduction over the latest state-of-the-art method MutexMatch on CIFAR-10 with 2 labels per class and exhibits robustness in the scenario of mismatched distribution.