FedDA: Resource-adaptive federated learning with dual-alignment aggregation optimization for heterogeneous edge devices

Abstract

Federated learning (FL) is an emerging distributed learning paradigm that allows multiple clients to collaborate on training a global model without sharing their local data. However, in practical heterogeneous edge device scenarios, FL faces the challenges of system heterogeneity and data heterogeneity, which leads to unfair participation and degraded global model performance. In this paper, we introduce FedDA, a resource-adaptive FL framework, which adapts to the client’s computing resources by assigning heterogeneous models of different sizes. To improve the performance of heterogeneous model aggregation and adjust to non-independent and identically distributed (non-i.i.d.) data, we propose a dual-alignment aggregation optimization method, i.e., parameter feature space alignment and output space alignment. Specifically, FedDA exploits the permutation symmetry property of weight space to permutate the model parameter positions via an adaptive layer-wise matching method, thereby aligning models with significant deviations in parameter feature space. FedDA mitigates the imbalance in parameter quantity between smaller and larger models through parameter expansion. Additionally, FedDA maps client labels into a uniform embedding space through output space alignment, thus reducing model parameter deviations due to non-i.i.d. data without additional client-side computing overhead. We evaluate the performance of FedDA on benchmark datasets, including FashionMNIST, CIFAR10, CIFAR100 and AGNews. Experimental results demonstrate that FedDA achieves up to 8.71% improvement in model accuracy compared to baseline methods, highlighting its effectiveness in addressing the challenges of heterogeneity.