基于多智能体深度强化学习的车辆边缘计算协同任务卸载策略

IF 6.2 2区 计算机科学 Q1 COMPUTER SCIENCE, THEORY & METHODS
Yuya Cui , Degan Zhang , Honghu Li , Hao Qiang , Haitao Zhao
{"title":"基于多智能体深度强化学习的车辆边缘计算协同任务卸载策略","authors":"Yuya Cui ,&nbsp;Degan Zhang ,&nbsp;Honghu Li ,&nbsp;Hao Qiang ,&nbsp;Haitao Zhao","doi":"10.1016/j.future.2025.107950","DOIUrl":null,"url":null,"abstract":"<div><div>Vehicular Edge Computing (VEC) and Vehicle- to-Vehicle (V2V) offloading can significantly reduce in-vehicle task latency. This paper investigates a cooperative task offloading strategy in VEC, where latency-sensitive and computation -intensive tasks can be offloaded to Road Side Units (RSUs) using 5 G connectivity. Additionally, these tasks can be shared among nearby vehicles through V2V links. Joint VEC and V2V cooperative offloading can not only minimizes task execution delays but also prevents network congestion. When vehicles are in motion, dynamic migration of computation tasks is necessary to maintain service continuity. We propose a two-phase distributed task offloading and migration strategy for multiple vehicles. In the first phase, vehicles select the optimal service vehicle based on inter-vehicle link quality and offloading willingness. In the second phase, to minimize system cost, we introduce a multi-agent reinforcement learning (MARL) based distributed task offloading and migration strategy. This strategy allows vehicles to choose the optimal edge node in a dynamic environment without fully offloading information. Moreover, we implement a counterfactual multi- agent (COMA) reinforcement learning approach to address the inefficiency caused by the credit allocation problem in multi-agent systems. Extensive evaluations demonstrate that the algorithm proposed in this paper perform better in terms of average system latency and overall task completion rate. Compared with related scheme, the proposed method can reduce latency by up to 54 % and improve task completion rate by up to 15 % in different scenarios.</div></div>","PeriodicalId":55132,"journal":{"name":"Future Generation Computer Systems-The International Journal of Escience","volume":"174 ","pages":"Article 107950"},"PeriodicalIF":6.2000,"publicationDate":"2025-06-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Cooperative Task Offloading Strategy for Vehicular Edge Computing Based on Multi-Agent Deep Reinforcement Learning\",\"authors\":\"Yuya Cui ,&nbsp;Degan Zhang ,&nbsp;Honghu Li ,&nbsp;Hao Qiang ,&nbsp;Haitao Zhao\",\"doi\":\"10.1016/j.future.2025.107950\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><div>Vehicular Edge Computing (VEC) and Vehicle- to-Vehicle (V2V) offloading can significantly reduce in-vehicle task latency. This paper investigates a cooperative task offloading strategy in VEC, where latency-sensitive and computation -intensive tasks can be offloaded to Road Side Units (RSUs) using 5 G connectivity. Additionally, these tasks can be shared among nearby vehicles through V2V links. Joint VEC and V2V cooperative offloading can not only minimizes task execution delays but also prevents network congestion. When vehicles are in motion, dynamic migration of computation tasks is necessary to maintain service continuity. We propose a two-phase distributed task offloading and migration strategy for multiple vehicles. In the first phase, vehicles select the optimal service vehicle based on inter-vehicle link quality and offloading willingness. In the second phase, to minimize system cost, we introduce a multi-agent reinforcement learning (MARL) based distributed task offloading and migration strategy. This strategy allows vehicles to choose the optimal edge node in a dynamic environment without fully offloading information. Moreover, we implement a counterfactual multi- agent (COMA) reinforcement learning approach to address the inefficiency caused by the credit allocation problem in multi-agent systems. Extensive evaluations demonstrate that the algorithm proposed in this paper perform better in terms of average system latency and overall task completion rate. Compared with related scheme, the proposed method can reduce latency by up to 54 % and improve task completion rate by up to 15 % in different scenarios.</div></div>\",\"PeriodicalId\":55132,\"journal\":{\"name\":\"Future Generation Computer Systems-The International Journal of Escience\",\"volume\":\"174 \",\"pages\":\"Article 107950\"},\"PeriodicalIF\":6.2000,\"publicationDate\":\"2025-06-06\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Future Generation Computer Systems-The International Journal of Escience\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0167739X25002456\",\"RegionNum\":2,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"COMPUTER SCIENCE, THEORY & METHODS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Future Generation Computer Systems-The International Journal of Escience","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0167739X25002456","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, THEORY & METHODS","Score":null,"Total":0}
引用次数: 0

摘要

车辆边缘计算(VEC)和车对车(V2V)卸载可以显著降低车内任务延迟。本文研究了VEC中的协作任务卸载策略,其中延迟敏感和计算密集型任务可以通过5g连接卸载到路旁单元(rsu)。此外,这些任务可以通过V2V链路在附近的车辆之间共享。VEC和V2V联合卸载不仅可以最大限度地减少任务执行延迟,还可以防止网络拥塞。当车辆处于运动状态时,为了保证业务的连续性,需要对计算任务进行动态迁移。提出了一种针对多车辆的两阶段分布式任务卸载和迁移策略。在第一阶段,车辆根据车辆间链路质量和卸载意愿选择最优服务车辆。在第二阶段,为了最小化系统成本,我们引入了基于多智能体强化学习(MARL)的分布式任务卸载和迁移策略。该策略允许车辆在不完全卸载信息的情况下,在动态环境中选择最优边缘节点。此外,我们实现了一种反事实多智能体(COMA)强化学习方法来解决多智能体系统中由于信用分配问题而导致的低效率问题。大量的评估表明,本文提出的算法在平均系统延迟和整体任务完成率方面表现更好。与相关方案相比,该方法在不同场景下的延迟降低高达54%,任务完成率提高高达15%。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Cooperative Task Offloading Strategy for Vehicular Edge Computing Based on Multi-Agent Deep Reinforcement Learning
Vehicular Edge Computing (VEC) and Vehicle- to-Vehicle (V2V) offloading can significantly reduce in-vehicle task latency. This paper investigates a cooperative task offloading strategy in VEC, where latency-sensitive and computation -intensive tasks can be offloaded to Road Side Units (RSUs) using 5 G connectivity. Additionally, these tasks can be shared among nearby vehicles through V2V links. Joint VEC and V2V cooperative offloading can not only minimizes task execution delays but also prevents network congestion. When vehicles are in motion, dynamic migration of computation tasks is necessary to maintain service continuity. We propose a two-phase distributed task offloading and migration strategy for multiple vehicles. In the first phase, vehicles select the optimal service vehicle based on inter-vehicle link quality and offloading willingness. In the second phase, to minimize system cost, we introduce a multi-agent reinforcement learning (MARL) based distributed task offloading and migration strategy. This strategy allows vehicles to choose the optimal edge node in a dynamic environment without fully offloading information. Moreover, we implement a counterfactual multi- agent (COMA) reinforcement learning approach to address the inefficiency caused by the credit allocation problem in multi-agent systems. Extensive evaluations demonstrate that the algorithm proposed in this paper perform better in terms of average system latency and overall task completion rate. Compared with related scheme, the proposed method can reduce latency by up to 54 % and improve task completion rate by up to 15 % in different scenarios.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
CiteScore
19.90
自引率
2.70%
发文量
376
审稿时长
10.6 months
期刊介绍: Computing infrastructures and systems are constantly evolving, resulting in increasingly complex and collaborative scientific applications. To cope with these advancements, there is a growing need for collaborative tools that can effectively map, control, and execute these applications. Furthermore, with the explosion of Big Data, there is a requirement for innovative methods and infrastructures to collect, analyze, and derive meaningful insights from the vast amount of data generated. This necessitates the integration of computational and storage capabilities, databases, sensors, and human collaboration. Future Generation Computer Systems aims to pioneer advancements in distributed systems, collaborative environments, high-performance computing, and Big Data analytics. It strives to stay at the forefront of developments in grids, clouds, and the Internet of Things (IoT) to effectively address the challenges posed by these wide-area, fully distributed sensing and computing systems.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信