Continual and wisdom learning for federated learning: A comprehensive framework for robustness and debiasing

Abstract

Federated Learning (FL) has transformed decentralized machine learning, however it remains has concerns with noisy labeled data, diverse clients, and sparse datasets, especially in delicate fields like healthcare. To address these issues, this study introduces a robust FL framework that integrates advanced Continual Learning (CL) and Wisdom Learning (WL) techniques. Elastic Weight Consolidation (EWC) prevents catastrophic forgetting by penalizing deviations from critical weights, while Progressive Neural Networks (PNN) leverage modular architectures with lateral connections to enable knowledge transfer across tasks and isolate client-specific biases. WL incorporates consensus-based aggregation, dynamic model distillation, and adaptive ensemble learning to enhance model robustness against noisy updates and biased data distributions. The framework is rigorously validated on benchmark medical imaging datasets, including ADNI, BraTS, PathMNIST, BreastMNIST, and ChestMNIST, demonstrating significant improvements in fairness metrics, with up to a 94.3% reduction in bias (Demographic Parity) and a 92.7% improvement in accuracy fairness (Accuracy Parity). These results establish the effectiveness of the proposed approach in achieving stable, equitable, and high-performing global models under challenging FL conditions characterized by dynamic client settings, label noise, and class imbalance.