FedPKR: Federated Learning With Non-IID Data via Periodic Knowledge Review in Edge Computing

Abstract

Federated learning is a distributed learning paradigm, which is usually combined with edge computing to meet the joint training of IoT devices. A significant challenge in federated learning lies in the statistical heterogeneity, characterized by non-independent and identically distributed (non-IID) local data across diverse parties. This heterogeneity can result in inconsistent optimization within individual local models. Although previous research has endeavored to tackle issues stemming from heterogeneous data, our findings indicate that these attempts have not yielded high-performance neural network models. To overcome this fundamental challenge, we introduce the framework called FedPKR in this paper, which facilitates efficient federated learning through knowledge review. The core principle of FedPKR involves leveraging the knowledge representation generated by the global and local model layers to conduct periodic layer-by-layer comparative learning in a reciprocal manner. This strategy rectifies local model training, leading to enhanced outcomes. Our experimental results and subsequent analysis substantiate that FedPKR effectively augments model accuracy in image classification tasks, meanwhile demonstrating resilience to statistical heterogeneity across all participating entities.