Robust Federated Learning For Image Semantic Transmission Under Byzantine Attacks

Semantic communication provides an efficient solution for image transmission, leveraging the advancements in joint source and channel coding facilitated by deep neural networks (DNNs). To address the limited communication and computing resources in edge devices, federated learning (FL) has emerged as a promising approach for collaboratively training semantic encoder/decoder models of semantic communication systems. However, FL-enabled semantic communication systems are still susceptible to malicious attacks, such as raw data tampering and incorrect transmission of semantic models, which pose significant security and integrity risks. In this paper, we propose a robust FL framework for image semantic transmission (IST) to defend against Byzantine attacks, i.e., data and model poisoning attacks, enabling more robust model training with fewer user data fitting. In the proposed framework, we design an end-to-end IST model based on convolutional neural networks (CNNs) to process semantic information. Then, we investigate the impact of Byzantine attacks on image recognition tasks in semantic communication systems and propose an improved Weiszfeld algorithm for model aggregation at the edge server, effectively mitigating the effects of Byzantine attacks on the IST model training process. Simulation results show that the proposed robust FL framework can adaptively select reliable users for aggregation while rejecting adversarial and low-quality users, thereby ensuring the robustness of semantic communication systems for image transmission.