ByFeDis: Toward Byzantine-Robust Heterogeneous Federated Learning via Knowledge Distillation in Railway Intelligent Control Systems

Abstract

Railway intelligent train control system must accommodate a wide range of scenarios due to the extensive variation in its operational environment. Federated learning based on knowledge distillation offers an efficient training paradigm for such systems by supporting model heterogeneity and reducing communication overhead. However, conventional federated knowledge distillation approaches often exhibit limited robustness against untrusted clients and do not facilitate continuous learning. In this paper, we design a Byzantine-robust federated knowledge distillation scheme, named ByFeDis, which mitigates the impact of malicious clients by removing outliers and low-value learning data, thereby improving learning efficiency. Additionally, we enhance the scalability of federated knowledge distillation in an improved scheme ByFeDis*, which is built on ByFeDis and enables continuous training for newly joined clients. Specifically, ByFeDis measures the soft labels uploaded by clients using Euclidean distance to identify poisoned soft labels that disrupt the training process. These abnormal soft labels include not only malicious perturbed samples that significantly deviate from the distribution of the true labels, but also speculative soft labels that are overly close to the true labels. To further enhance privacy protection, we employ fully homomorphic encryption to enable secure aggregation, thereby safeguarding sensitive client information. Theoretical analysis confirms that our scheme effectively preserves privacy while reducing communication complexity compared with existing approaches. Extensive experiments across multiple datasets demonstrate that ByFeDis achieves, on average, a 2% improvement in accuracy over other Byzantine-robust schemes, while ByFeDis* further enhances model accuracy by 3% and reduces premature convergence by up to 10 training rounds in subsequent learning.