FedDD: Combating Federated Noisy Label Learning With Intra- and Inter-Distributional Difference

Abstract

Federated learning (FL) enables multiple parties to collaboratively train a global machine learning model without sharing local data. While FL has made awe-inspiring advances, most existing studies assume fully clean data labels on clients, which rarely holds in real-world scenarios where label noise is prevalent and detrimental to FL performance. Prior approaches typically rely on the small-loss criterion to identify noisy instances, which has been proven to struggle in highly heterogeneous settings. To overcome this, we propose a robust framework called FedDD that tackles label noise in FL by leveraging intra- and inter-distributional differences. Instead, FedDD employs an intra-difference measure, which utilizes the intra-logit variance to preliminarily identify the clean subset. Subsequently, it refines the data by introducing an inter-distributional difference metric based on prediction discrepancies between local and global models. To further utilize identified noisy examples, a novel federated semi-supervised learning paradigm is proposed that amalgamates knowledge from the global and local models to guide pseudo-labeling. Extensive experiments on benchmark datasets under both independent and identically distributed (IID) and non-IID settings demonstrate the superiority of FedDD over state-of-the-art methods.