FedStrag: Straggler-aware federated learning for low resource devices

Abstract

Federated Learning (FL) has become a popular training paradigm in recent years. However, stragglers are critical bottlenecks in an Internet of Things (IoT) network while training. These nodes produce stale updates to the server, which slow down the convergence. In this paper, we studied the impact of the stale updates on the global model, which is observed to be significant. To address this, we propose a weighted averaging scheme, FedStrag, that optimizes the training with stale updates. The work is focused on training a model in an IoT network that has multiple challenges, such as resource constraints, stragglers, network issues, device heterogeneity, etc. To this end, we developed a time-bounded asynchronous FL paradigm that can train a model on the continuous inflow of data in the edge-fog-cloud continuum. To test the FedStrag approach, a model is trained with multiple stragglers scenarios on both Independent and Identically Distributed (IID) and non-IID datasets on Raspberry Pis. The experiment results suggest that the FedStrag outperforms the baseline FedAvg in all possible cases.