Adaptive layer-wise personalized federated learning via dual delay update in future communication networks

The future communication networks refers to large-scale mass-connected networks consisting of billions of cloud, edge, and end devices, which are expected to support the ever-growing communication demands. In the future communication networks, billions of end devices generate massive amount of data that needs to be processed and analyzed (e.g., model training). Artificial Intelligence of Things (AIoT) is a groundbreaking technology that leverages artificial intelligence models to process and analyze data generated by a large number of internet of things devices. As an emerging AIoT method, personalized Federated Learning (pFL) has emerged prominently in distributed model training using massive data from the future communication networks. However, it is challenging to accomplish high-performance and communication-efficient model training by existing pFL methods in the future communication networks, due to the following limitations. a) Dynamic role differences in each layer of a multi-layer model are neglected, leading to poor accuracy in customized models deployed on end devices. b) Owing to numerous end devices in the future communication networks, the communication frequency between a cloud server and end devices is extremely high in each communication round, resulting in expensive communication cost. To solve these two limitations, this paper presents a novel pFL framework for distributed model training in the future communication networks, called Adaptive Layer-wise personalized Federated Learning via Dual Delay Update (ALpFLDDU). First, in end devices, a layer-wise aggregation scheme based on an adaptive weight calculation mechanism is designed to capture the dynamic role differences of model layers. Second, in each communication round, we develop a dual delay update strategy to reduce communication frequency between a cloud server and end devices while ensuring model performance. Simulation experiments on text and image classification datasets are conducted. The experimental results show that ALpFLDDU realizes higher classification precision and lower communication cost than advanced pFL benchmarks on various classification tasks.