Hybrid Fusion Learning: A Hierarchical Learning Model For Distributed Systems

Abstract

Federated and fusion learning methods are state-of-the-art distributed learning approaches which enable model training without collecting private data from users. While federated learning involves lower computation cost as compared to fusion learning, the overall communication cost is higher due to a large number of communication rounds between the clients and the server. On the other hand, fusion learning reduces the overall communication cost by sending distributions of features and model parameters using only one communication round but suffers from high computation cost as it needs to find the distributions of features at the client. This paper presents hybrid fusion learning, a system that leverages hierarchical client-edge-cloud architecture and builds a deep learning model by integrating both fusion and federated learning methods. Our proposed approach uses fusion learning between the client and the edge layer to minimise the communication cost whereas it uses federated learning between the edge and the cloud layer to minimise the computation cost. Our results show that the proposed hybrid fusion learning can significantly reduce the total time taken to train the model with a small drop of around 2% in accuracies as compared to the other two algorithms. Specifically, our results show that fusion and federated learning algorithms take up to 26.28% and 9.74% higher average total time to build the model, respectively, than the proposed hybrid fusion learning approach.