Digital Twin-Empowered Federated Incremental Learning for Non-IID Privacy Data

Abstract

Federated learning (FL) has emerged as a compelling distributed learning paradigm without sharing local original data. However, with ubiquitous non-independent and identically distributed (non-IID) privacy data, the FL suffers from severe performance loss and the privacy leakage by inference attacks. Existing solutions lack a cohesive framework with theoretical support, and their performance optimization and privacy protection are inter-inhibitive or high-cost. In this paper, we propose a digital twin (DT)-empowered federated incremental learning method to tackle the above challenges. First, we construct a DT-empowered federated incremental learning model to achieve cooperative awareness of performance and privacy-preservation. Second, a diffusion model-based selective data synthesis method is designed to provide auxiliary data for FL, it can avoid unnecessary overhead while ensuring the quality of synthetic samples under non-IID. Besides, it alleviates the negative impact of non-IID by allocating a class-balanced sub-dataset to each DT with IID setting. Third, we develop a DT-empowered alternating incremental learning method initiatively, under the premise of ensuring the confidentiality of original dataset, it can achieve efficient FL performance under non-IID with a small amount of synthetic samples. Furthermore, in order to estimate the contribution of each local model accurately, we investigate a comentropy-based federated aggregation strategy, which can obtain a superior global model. By sufficient theoretical analysis, we prove that the proposed methodology can achieve consistent enhancement of performance and privacy-preservation. Simultaneously, the experiments demonstrate that our methodology has efficient privacy-preserving property, it also outperforms other benchmarks on the accuracy and stability of the global model, especially in highly heterogeneous scenarios.