A Novel Prediction Technique for Federated Learning

Abstract

Researchers have studied how to improve Federated Learning (FL) in various areas, such as statistical and system heterogeneity, communication cost, and privacy. So far, most of the proposed solutions are either very tied to the application context or complex to be broadly reproduced in real-life applications involving humans. Developing modular solutions that can be leveraged by the vast majority of FL structures and are independent of the application people use is the new research direction opened by this paper. In this work, we propose a plugin (named FedPredict) to address three problems simultaneously: data heterogeneity, low performance of new/untrained and/or outdated clients, and communication cost. We do so mainly by combining global and local parameters (which brings generalization and personalization) in the inference step while adapting layer selection and matrix factorization techniques to reduce the downlink communication cost (server to client). Due to its simplicity, it can be applied to federated learning of different number of topologies. Results show that adding the proposed plugin to a given FL solution can significantly reduce the downlink communication cost by up to 83.3% and improve accuracy by up to 304% compared to the original solution.