Sym-CS-HFL: A secure and efficient solution for privacy-preserving heterogeneous federated learning

Abstract

In the era of big data, deep learning models play a crucial role in identifying underlying patterns within data. However, the need for large volumes of training data, often scattered across various organizations with privacy constraints, poses a significant challenge. Federated Learning (FL) addresses this by enabling the collaborative training of models without sharing the underlying data. Despite its promise, FL encounters challenges with model privacy leakage and computational overhead, particularly when dealing with non-identically distributed (Non-IID) data. To overcome these challenges, we introduce Sym-CS-HFL, a novel Privacy-Preserving Federated Learning (PPFL) framework that combines Symmetric Homomorphic Encryption with a Local Adaptive Aggregation (LAA) scheme. Our approach minimizes the reliance on asymmetric keys, simplifying the encryption process and reducing computational overhead. We implement a DCT-Neural Network Compressive Sensing Scheme to decrease communication costs substantially. Furthermore, the LAA scheme addresses the heterogeneity in Non-IID data, enhancing model convergence and accuracy. Our experiments on diverse datasets, including MNIST, FashionMNIST, CIFAR-10/100, and AG News, demonstrate that Sym-CS-HFL achieves a Top-3 test accuracy while significantly reducing communication overhead by $15.2\times$ to $74\times$ compared to existing HE schemes. The computational overhead is also reduced, with training times only $1.1\times$ to $1.8\times$ that of plaintext training. These results underscore Sym-CS-HFL’s effectiveness in maintaining high performance and privacy in PPFL.