FedSam: Enhancing federated learning accuracy with differential privacy and data heterogeneity mitigation

Abstract

A large-scale model is typically trained on an extensive dataset to update its parameters and enhance its classification capabilities. However, directly using such data can raise significant privacy concerns, especially in the medical field, where datasets often contain sensitive patient information. Federated Learning (FL) offers a solution by enabling multiple parties to collaboratively train a high-performance model without sharing their raw data. Despite this, during the federated training process, attackers can still potentially extract private information from local models. To bolster privacy protections, Differential Privacy (DP) has been introduced to FL, providing stringent safeguards. However, the combination of DP and data heterogeneity can often lead to reduced model accuracy. To tackle these challenges, we introduce a sampling-memory mechanism, FedSam, which improves the accuracy of the global model while maintaining the required noise levels for differential privacy. This mechanism also mitigates the adverse effects of data heterogeneity in heterogeneous federated environments, thereby improving the global model’s overall performance. Experimental evaluations on datasets demonstrate the superiority of our approach. FedSam achieves a classification accuracy of 95.03%, significantly outperforming traditional DP-FedAvg (91.74%) under the same privacy constraints, highlighting FedSam’s robustness and efficiency.