Federated learning with empirical insights: Leveraging gradient historical experiences for performance fairness

Performance fairness has always been a key issue in federated learning (FL), however, the pursuit of performance consistency can lead to a trade-off where the accuracy of well-performing clients is compromised to enhance the accuracy of poor-performing clients. To ensure equitable treatment and unbiased outcomes for all participants in the FL process, we propose FedMH, a fair and fast multi-gradient descent federated learning algorithm with reinforced gradient historical empirical information. We have conducted a theoretical analysis of FedMH from the perspectives of fairness and convergence. Extensive experiments are performed on four federated datasets, revealing significant improvements achieved by FedMH compared to state-of-the-art baselines. Moreover, the experimental findings highlight FedMH’s superior performance in fine-grained classification problems when compared to existing advanced baselines. In brief, the proper utilization of gradient historical empirical information helps improve the effectiveness and fairness of FL, making it more suitable for large-scale and heterogeneous distributed environments.