Advance-FL: A3C-Based Adaptive Asynchronous Online Federated Learning for Vehicular Edge Cloud Computing Networks

Abstract

Enhancing autonomous driving through Federated Learning (FL) in Intelligent Connected Vehicles (ICVs) confronts challenges like limited scalability of central management, computational strains on diverse ICVs, and inefficiencies due to stragglers. This paper presents $\mathit{Advance\hbox{-}FL}$, a deep reinforcement learning based $\underline{\mathit{A}}$$\underline{\mathit{d}}$apti$\underline{\mathit{v}}$e $\underline{\mathit{a}}$sy$\underline{\mathit{nc}}$hronous onlin$\underline{\mathit{e}}$ $\underline{\mathit{F}}$ederated $\underline{\mathit{L}}$earning with a computation offloading assisted framework for vehicular edge cloud computing networks to mitigate the above challenges. Innovatively, $\mathit{Advance\hbox{-}FL}$ incorporates the concept of “straggler rate”, a metric originally introduced in this study to quantify the degree of lag in participant computation and training, thus enabling targeted mitigation strategies. By employing an asynchronous advantage actor-critic approach for adaptive data offloading and dynamic local iteration adjustments, $\mathit{Advance\hbox{-}FL}$ effectively alleviates computation resource shortages and harmonizes the balance between model accuracy and straggler impact by dynamically managing the straggler rate. Critical findings include over 62% reduction in training times and a 2%$\sim$9% increase in model accuracy across varying non-IID data scenarios and reducing training time by more than 6 times as the number of ICVs increases, compared to prevailing methods. Additionally, experiments in both static and dynamic test-bed further validate $\mathit{Advance\hbox{-}FL}$’s superior scalability and robustness over state-of-the-art approaches, particularly in maintaining high performance under straggler effects, and showcasing robust adaptability across long-term operations, large-scale datasets and abnormal situations.