{"title":"Risk-Aware Reinforcement Learning-Based Federated Learning for IoV Systems","authors":"Xiaozhen Lu;Zhibo Liu;Yuhan Chen;Liang Xiao;Wei Wang;Qihui Wu","doi":"10.1109/TMC.2024.3447034","DOIUrl":null,"url":null,"abstract":"Federated learning (FL) that improves data privacy reduces the computational overhead for Internet of Vehicles (IoV) systems but has difficulty in defending against selfish attacks due to the restricted quality of service requirements and the high mobility of vehicles. In this paper, we design a risk-aware hierarchical reinforcement learning-based FL framework for IoV to resist selfish attacks. By designing a two-level hierarchical policy selection module that consists of two deep neural networks, this framework divides the training policy into two sub-policies, i.e., the selection of FL participants and the corresponding local training data size, which are chosen based on the previous training performance and vehicle participation performance. This framework designs a risk-aware safety guide to avoid dangerous states such as local task failure resulting from risky training policies. Specifically, the guide uses a warning signal to evaluate the short-term risk of each state-action pair, applies an R-network to estimate the long-term risks for modifying the chosen training policy, and designs a punishment function for the modified training policy to revise the immediate reward to further enhance the safe exploration. We analyze the convergence performance and computational complexity of our scheme. Experimental results on MNIST, CIFAR-10, and Stanford Cars datasets verify the effectiveness of our scheme, including the global model accuracy, training latency, detection success rate, and convergence speed compared with the benchmarks FedAvg, MFL, DQNPS, and SHRL.","PeriodicalId":50389,"journal":{"name":"IEEE Transactions on Mobile Computing","volume":"23 12","pages":"14672-14688"},"PeriodicalIF":7.7000,"publicationDate":"2024-08-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Mobile Computing","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10643307/","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0
Abstract
Federated learning (FL) that improves data privacy reduces the computational overhead for Internet of Vehicles (IoV) systems but has difficulty in defending against selfish attacks due to the restricted quality of service requirements and the high mobility of vehicles. In this paper, we design a risk-aware hierarchical reinforcement learning-based FL framework for IoV to resist selfish attacks. By designing a two-level hierarchical policy selection module that consists of two deep neural networks, this framework divides the training policy into two sub-policies, i.e., the selection of FL participants and the corresponding local training data size, which are chosen based on the previous training performance and vehicle participation performance. This framework designs a risk-aware safety guide to avoid dangerous states such as local task failure resulting from risky training policies. Specifically, the guide uses a warning signal to evaluate the short-term risk of each state-action pair, applies an R-network to estimate the long-term risks for modifying the chosen training policy, and designs a punishment function for the modified training policy to revise the immediate reward to further enhance the safe exploration. We analyze the convergence performance and computational complexity of our scheme. Experimental results on MNIST, CIFAR-10, and Stanford Cars datasets verify the effectiveness of our scheme, including the global model accuracy, training latency, detection success rate, and convergence speed compared with the benchmarks FedAvg, MFL, DQNPS, and SHRL.
期刊介绍:
IEEE Transactions on Mobile Computing addresses key technical issues related to various aspects of mobile computing. This includes (a) architectures, (b) support services, (c) algorithm/protocol design and analysis, (d) mobile environments, (e) mobile communication systems, (f) applications, and (g) emerging technologies. Topics of interest span a wide range, covering aspects like mobile networks and hosts, mobility management, multimedia, operating system support, power management, online and mobile environments, security, scalability, reliability, and emerging technologies such as wearable computers, body area networks, and wireless sensor networks. The journal serves as a comprehensive platform for advancements in mobile computing research.