Autoencoder-based decentralized federated learning for efficient communication

Abstract

Decentralized federated learning (DFL) has emerged as a solution for traditional federated learning’s limitations, such as network bottlenecks and single-point failure, by enabling direct communication between nodes and eliminating the reliance on a central server. However, DFL still encounters challenges like increased communication costs as the number of participating nodes increases, amplifying the need for efficient compression techniques. Moreover, the increasing complexity of models, including vision, language, and generative models (e.g., GPT), further underscores this necessity due to their large parameter sizes. To address the communication cost-related issues in DFL, this study introduces Autoencoder-based Decentralized Federated Learning (AEDFL), which leverages autoencoders to compress model updates before transmission, allowing them to be reconstructed at the receiving end with high fidelity and minimal loss of accuracy. We conduct comprehensive experiments using two models, SqueezeNet and DenseNet, on three benchmark datasets: CIFAR-10 (under both IID and non-IID settings), FashionMNIST, and CIFAR-100. The results demonstrate that AEDFL achieves up to 122x compression with negligible accuracy degradation, showcasing its effectiveness in balancing communication efficiency and model performance across varying model sizes and dataset complexities.