Accelerating personalized federated learning via dynamic gradient substitution and client selection

Personalized federated learning (PFL) has gained widespread attention for its ability to preserve privacy and adapt to user-specific characteristics. Among the leading PFL methods, meta-learning based algorithms like Per-FedAvg offer a unified framework of gradient updates for all clients, eliminating the necessity of personalized model architectures that are common in other PFL approaches. However, their computation inefficiency and challenges in accommodating system heterogeneity are under-explored. This work proposes pFedSara, a novel PFL framework that accelerates the training of a target PFL method, Per-FedAvg, by exploiting the lightweight, vanilla FL algorithm, FedAvg. Instead of fervently creating marginally altered approaches, pFedSara is the first that strategically reuses and blends existing techniques for PFL training, navigating the runtime-accuracy trade-off, and it offers a comprehensive theoretical analysis. Specifically, it leverages dynamic gradient substitution and client selection by assessing runtime, loss, and gradient similarity between FedAvg and Per-FedAvg, the two candidate local update methods for each client. Additionally, it incorporates gradient scaling to accommodate incomplete Per-FedAvg computations that cannot be replaced by FedAvg, eliminating additional biases. A novel convergence analysis is provided, quantifying the biases introduced by both heterogeneous data and our employed hybrid update methods for computation speed-up. Extensive experiments demonstrate that pFedSara achieves superior training efficiency compared with state-of-the-art PFL methods.