Affinity-aware uncertainty quantification for learning with noisy labels

Abstract

Training deep neural networks (DNNs) with noisy labels is a challenging task that significantly degenerates the model’s performance. Most existing methods mitigate this problem by identifying and eliminating noisy samples or correcting their labels according to statistical properties like confidence values. However, these methods often overlook the impact of inherent noise, such as sample quality, which can mislead DNNs to focus on incorrect regions, adulterate the softmax classifier, and generate low-quality pseudo-labels. In this paper, we propose a novel Affinity-aware Uncertainty Quantification (AUQ) framework to explore the perception ambiguity and rectify the salient bias by quantifying the uncertainty. Concretely, we construct the dynamic prototypes to represent intra-class semantic spaces and estimate the uncertainty based on sample-prototype pairs, where the observed affinities between sample-prototype pairs are converted to probabilistic representations as the estimated uncertainty. Samples with higher uncertainty are likely to be hard samples and we design an uncertainty-aware loss to emphasize the learning from those samples with high uncertainty, which helps DNNs to gradually concentrate on the critical regions. Besides, we further utilize sample-prototype affinities to adaptively refine pseudo-labels, enhancing the quality of supervisory signals for noisy samples. Extensive experiments conducted on the CIFAR-10, CIFAR-100 and Clothing1M datasets demonstrate the efficacy and effectiveness of AUQ. Notably, we achieve an average performance gain of 0.4 % on CIFAR-10 and a substantial average improvement of 2.3 % over the second-best method on the more challenging CIFAR-100 dataset. Moreover, there is a 0.6 % improvement over the sub-optimal method on Clothing1M. These results validate AUQ’s capability in enhancing DNN robustness against noisy labels.