FedWAPR: Bridging theory and practice in probability-driven weighted aggregation for federated learning

Abstract

Federated Learning (FL) is a machine learning paradigm emphasizing data privacy, widely adopted for handling sensitive data. Federated Averaging (FedAvg) is the most commonly implemented FL aggregation technique due to its simplicity and effectiveness. However, FedAvg suffers from information loss during the aggregation stage. This study theoretically and empirically analyzes the Weighted Aggregation via Probability-based Ranking (FedWAPR) technique, an enhancement to FedAvg that retains its simplicity while addressing its limitations. FedWAPR employs a weighted aggregation strategy based on Log-Cauchy and Exponential probability density functions, assigning weights to local models based on their performance. This approach ensures accurate aggregation that reflects the contributions of individual clients. FedWAPR was tested across various model architectures, including Dense Neural Networks, Long Short-Term Memory networks, and Convolutional Neural Networks with results showing performance equal to or surpassing FedAvg. The Log-Cauchy and Exponential distribution functions allow customization of aggregation based on the number of participating clients, with exponential distribution excelling in smaller client setups and Log-Cauchy in larger ones. FedWAPR’s ability to integrate with advanced aggregation techniques like FedProx, makes it a robust solution to enhance FL. Additionally, a theoretical analysis confirms the convergence of FedWAPR under standard FL assumptions and thereby ensuring method’s robustness and reliability.