A communication-efficient federated learning approach via dynamic mutual distillation for image recognition

Abstract

Federated learning is a promising approach to protect data privacy in the image recognition field, enabling collaborative model training across distributed edge participants without compromising local data. However, the privacy-preserving feature comes at the cost of significant communication overhead due to frequent model parameter exchanges. In the current edge computing environment, clients are usually deployed on edge devices with limited bandwidth, the communication delay greatly influences the training efficiency of federated learning. Hence, the paper proposes a communication-efficient federated learning approach FedDMS based on mutual distillation and dynamic client selection for image recognition. It improves the convergence efficiency through client-side dynamic distillation and increases task accuracy through fine-tuning. In addition, the server adaptively selects participation clients through periodic gradient evaluation, thereby reducing the communication overhead. FedDMS is evaluated from the aspects of performance and parameter sensitivity on two public datasets. The experimental results show that compared with other federated algorithms, FedDMS can save 73% of communication costs, significantly improving efficiency. Furthermore, FedDMS’s performance remains stable in different network structures, demonstrating its strong adaptability and optimization potential. At a cost, it requires additional computing resources on the client side.