Federated or Split? A Performance and Privacy Analysis of Hybrid Split and Federated Learning Architectures

Abstract

Mobile phones, wearable devices, and other sensors produce every day a large amount of distributed and sensitive data. Classical machine learning approaches process these large datasets usually on a single machine, training complex models to obtain useful predictions. To better preserve user and data privacy and at the same time guarantee high performance, distributed machine learning techniques such as Federated and Split Learning have been recently proposed. Both of these distributed learning architectures have merits but also drawbacks. In this work, we analyze such tradeoffs and propose a new hybrid Federated Split Learning architecture, to combine the benefits of both in terms of efficiency and privacy. Our evaluation shows how Federated Split Learning may reduce the computational power required for each client running a Federated Learning and enable Split Learning parallelization while maintaining a high prediction accuracy with unbalanced datasets during training. Furthermore, FSL provides a better accuracy-privacy tradeoff in specific privacy approaches compared to Parallel Split Learning.