Hybrid Multi-Server Computation Offloading in Air–Ground Vehicular Networks Empowered by Federated Deep Reinforcement Learning

IF 6.7 2区计算机科学 Q1 ENGINEERING, MULTIDISCIPLINARY

IEEE Transactions on Network Science and Engineering Pub Date : 2024-07-23 DOI:10.1109/TNSE.2024.3432765

Xiaoqin Song;Quan Chen;Shumo Wang;Tiecheng Song;Lei Xu

{"title":"Hybrid Multi-Server Computation Offloading in Air–Ground Vehicular Networks Empowered by Federated Deep Reinforcement Learning","authors":"Xiaoqin Song;Quan Chen;Shumo Wang;Tiecheng Song;Lei Xu","doi":"10.1109/TNSE.2024.3432765","DOIUrl":null,"url":null,"abstract":"The proliferation of computation-intensive and delay-sensitive services in intelligent transportation systems, such as autonomous driving and vehicle-mounted infotainment services, presents a significant challenge for vehicular users (VUs) with limited resources. To address this issue, multi-access edge computing (MEC) has been considered a favorable solution to mitigate computation delay. This paper considers computation offloading for an air-ground integrated computing platform in vehicular networks. Specifically, we first propose a multi-agent twin delayed deep deterministic policy gradient (MATD3) algorithm to optimize the trajectory of UAVs. Then, an algorithm named federated upgraded dueling double deep Q network (FUD3QN) is proposed to meet quality of service (QoS) requirements. The algorithm allocates cross-domain resources after offloading decision-making, aiming to minimize delay and energy consumption while meeting reliability requirements, maximum tolerable delay, communication requirements, and computing limitations. Addressing the non-deterministic polynomial (NP)-hard problem, we employ a multi-agent federated learning and upgraded dueling double deep Q network algorithm (UD3QN) with centralized training and distributed execution. Simulation results illustrate that the MATD3-FUD3QN algorithm proposed significantly surpasses the baselines, highlighting the advantages of introducing UAVs to enhance transmission quality.","PeriodicalId":54229,"journal":{"name":"IEEE Transactions on Network Science and Engineering","volume":"11 6","pages":"5175-5189"},"PeriodicalIF":6.7000,"publicationDate":"2024-07-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Network Science and Engineering","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10607960/","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, MULTIDISCIPLINARY","Score":null,"Total":0}

引用次数: 0

Abstract

The proliferation of computation-intensive and delay-sensitive services in intelligent transportation systems, such as autonomous driving and vehicle-mounted infotainment services, presents a significant challenge for vehicular users (VUs) with limited resources. To address this issue, multi-access edge computing (MEC) has been considered a favorable solution to mitigate computation delay. This paper considers computation offloading for an air-ground integrated computing platform in vehicular networks. Specifically, we first propose a multi-agent twin delayed deep deterministic policy gradient (MATD3) algorithm to optimize the trajectory of UAVs. Then, an algorithm named federated upgraded dueling double deep Q network (FUD3QN) is proposed to meet quality of service (QoS) requirements. The algorithm allocates cross-domain resources after offloading decision-making, aiming to minimize delay and energy consumption while meeting reliability requirements, maximum tolerable delay, communication requirements, and computing limitations. Addressing the non-deterministic polynomial (NP)-hard problem, we employ a multi-agent federated learning and upgraded dueling double deep Q network algorithm (UD3QN) with centralized training and distributed execution. Simulation results illustrate that the MATD3-FUD3QN algorithm proposed significantly surpasses the baselines, highlighting the advantages of introducing UAVs to enhance transmission quality.

查看原文本刊更多论文

由联合深度强化学习驱动的空地车载网络中的混合多服务器计算卸载

在自动驾驶和车载信息娱乐服务等智能交通系统中，计算密集型和延迟敏感型服务的激增给资源有限的车辆用户（VUs）带来了巨大挑战。为解决这一问题，多访问边缘计算（MEC）被认为是缓解计算延迟的有利解决方案。本文考虑了车载网络中空地一体化计算平台的计算卸载问题。具体来说，我们首先提出了一种多代理双延迟深度确定性策略梯度（MATD3）算法来优化无人机的轨迹。然后，我们提出了一种名为联合升级对决双深 Q 网络（FUD3QN）的算法，以满足服务质量（QoS）要求。该算法在卸载决策后分配跨域资源，目的是在满足可靠性要求、最大可容忍延迟、通信要求和计算限制的同时，最大限度地减少延迟和能耗。为了解决这个非确定性多项式（NP）困难问题，我们采用了集中训练和分布式执行的多代理联合学习和升级版对决双深度 Q 网络算法（UD3QN）。仿真结果表明，所提出的 MATD3-FUD3QN 算法明显优于基线算法，凸显了引入无人机提高传输质量的优势。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

IEEE Transactions on Network Science and Engineering Engineering-Control and Systems Engineering

CiteScore

12.60

自引率

9.10%

发文量

393

期刊介绍： The proposed journal, called the IEEE Transactions on Network Science and Engineering (TNSE), is committed to timely publishing of peer-reviewed technical articles that deal with the theory and applications of network science and the interconnections among the elements in a system that form a network. In particular, the IEEE Transactions on Network Science and Engineering publishes articles on understanding, prediction, and control of structures and behaviors of networks at the fundamental level. The types of networks covered include physical or engineered networks, information networks, biological networks, semantic networks, economic networks, social networks, and ecological networks. Aimed at discovering common principles that govern network structures, network functionalities and behaviors of networks, the journal seeks articles on understanding, prediction, and control of structures and behaviors of networks. Another trans-disciplinary focus of the IEEE Transactions on Network Science and Engineering is the interactions between and co-evolution of different genres of networks.