Federated Learning: Optimizing Objective Function

Abstract

A universal server coordinates the training of a single model on a largely distributed network of computers in federated learning. This setting can easily be expanded to a multi-task learning system in order to manage real-world federated datasets with high statistical heterogeneity across devices. Federated learning is very useful as a framework for real-world data and federated multi-task learning has been applied to convex models. This research work discusses and evaluates possibility of sparser gradient changes to outperform the existing state-of-the-art for federated learning on real-world federated datasets as well as imputed data values. The experiments investigate the effect of rolling data or data randomization and adaptive global frequency update scheduling on the convergence of the federated learning algorithm. The results show that convergence speed and gradient curve are considerably affected by number of contact rounds between worker and aggregator and is unaffected by data heterogeneity or client sampling. The research is the core part of an extended experimental setup that will follow to better understand the behavior of distributed learning, by developing a simulation to track weights and loss function gradients during the training.