基于深度强化学习的移动车辆多跳任务卸载迁移与执行决策

IF 5.8 2区 计算机科学 Q1 TELECOMMUNICATIONS
Wenjie Zhou, Tian Zhang, Zekun Lu, Linbo Zhai
{"title":"基于深度强化学习的移动车辆多跳任务卸载迁移与执行决策","authors":"Wenjie Zhou,&nbsp;Tian Zhang,&nbsp;Zekun Lu,&nbsp;Linbo Zhai","doi":"10.1016/j.vehcom.2025.100950","DOIUrl":null,"url":null,"abstract":"<div><div>As the Internet of Things (IoT) drives the development of Vehicular Edge Computing (VEC), there is a surge in computational demand from emerging in-vehicle applications. Most existing studies do not fully consider the frequent changes in network topology under high mobility of vehicles and the underutilization of idle resources by single-hop offloading. To this end, we propose a task offloading scheme for vehicular edge computing based on multi-hop offloading. The scheme allows task vehicles to offload tasks to service vehicles with excess idle resources outside the communication range, and adapts to dynamic changes in network topology by introducing the concept of neighboring vehicle connection time. This study aims to minimize the delayed energy consumption utility value of the task under the conditions of satisfying the maximum task delay limit, vehicle computational and storage resource constraints. In response to this NP-hard problem, a two-stage reinforcement learning strategy MOCDD (combining Deep Q Network (DQN) and Deep Deterministic Policy Gradient (DDPG)) is proposed to divide the mixed action space into pure discrete and pure continuous action space to determine task migration, executive decision and vehicle transmission power. Simulation results verify the effectiveness of the proposed scheme.</div></div>","PeriodicalId":54346,"journal":{"name":"Vehicular Communications","volume":"55 ","pages":"Article 100950"},"PeriodicalIF":5.8000,"publicationDate":"2025-07-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Deep reinforcement learning based migration and execution decisions for multi-hop task offloading in mobile vehicle edge computing\",\"authors\":\"Wenjie Zhou,&nbsp;Tian Zhang,&nbsp;Zekun Lu,&nbsp;Linbo Zhai\",\"doi\":\"10.1016/j.vehcom.2025.100950\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><div>As the Internet of Things (IoT) drives the development of Vehicular Edge Computing (VEC), there is a surge in computational demand from emerging in-vehicle applications. Most existing studies do not fully consider the frequent changes in network topology under high mobility of vehicles and the underutilization of idle resources by single-hop offloading. To this end, we propose a task offloading scheme for vehicular edge computing based on multi-hop offloading. The scheme allows task vehicles to offload tasks to service vehicles with excess idle resources outside the communication range, and adapts to dynamic changes in network topology by introducing the concept of neighboring vehicle connection time. This study aims to minimize the delayed energy consumption utility value of the task under the conditions of satisfying the maximum task delay limit, vehicle computational and storage resource constraints. In response to this NP-hard problem, a two-stage reinforcement learning strategy MOCDD (combining Deep Q Network (DQN) and Deep Deterministic Policy Gradient (DDPG)) is proposed to divide the mixed action space into pure discrete and pure continuous action space to determine task migration, executive decision and vehicle transmission power. Simulation results verify the effectiveness of the proposed scheme.</div></div>\",\"PeriodicalId\":54346,\"journal\":{\"name\":\"Vehicular Communications\",\"volume\":\"55 \",\"pages\":\"Article 100950\"},\"PeriodicalIF\":5.8000,\"publicationDate\":\"2025-07-02\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Vehicular Communications\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S2214209625000774\",\"RegionNum\":2,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"TELECOMMUNICATIONS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Vehicular Communications","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2214209625000774","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"TELECOMMUNICATIONS","Score":null,"Total":0}
引用次数: 0

摘要

随着物联网(IoT)推动车辆边缘计算(VEC)的发展,新兴车载应用的计算需求激增。现有的研究大多没有充分考虑车辆高机动性下网络拓扑结构的频繁变化和单跳卸载对空闲资源的充分利用。为此,我们提出了一种基于多跳卸载的车辆边缘计算任务卸载方案。该方案允许任务车辆将任务卸载到通信范围外有多余空闲资源的服务车辆上,并通过引入相邻车辆连接时间的概念来适应网络拓扑的动态变化。本研究的目标是在满足最大任务延迟限制、车辆计算和存储资源约束的条件下,使任务的延迟能耗效用值最小化。针对这一NP-hard问题,提出了一种结合深度Q网络(Deep Q Network, DQN)和深度确定性策略梯度(Deep Deterministic Policy Gradient, DDPG)的两阶段强化学习策略MOCDD,将混合动作空间划分为纯离散和纯连续动作空间,以确定任务迁移、执行决策和车辆传输功率。仿真结果验证了该方案的有效性。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Deep reinforcement learning based migration and execution decisions for multi-hop task offloading in mobile vehicle edge computing
As the Internet of Things (IoT) drives the development of Vehicular Edge Computing (VEC), there is a surge in computational demand from emerging in-vehicle applications. Most existing studies do not fully consider the frequent changes in network topology under high mobility of vehicles and the underutilization of idle resources by single-hop offloading. To this end, we propose a task offloading scheme for vehicular edge computing based on multi-hop offloading. The scheme allows task vehicles to offload tasks to service vehicles with excess idle resources outside the communication range, and adapts to dynamic changes in network topology by introducing the concept of neighboring vehicle connection time. This study aims to minimize the delayed energy consumption utility value of the task under the conditions of satisfying the maximum task delay limit, vehicle computational and storage resource constraints. In response to this NP-hard problem, a two-stage reinforcement learning strategy MOCDD (combining Deep Q Network (DQN) and Deep Deterministic Policy Gradient (DDPG)) is proposed to divide the mixed action space into pure discrete and pure continuous action space to determine task migration, executive decision and vehicle transmission power. Simulation results verify the effectiveness of the proposed scheme.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Vehicular Communications
Vehicular Communications Engineering-Electrical and Electronic Engineering
CiteScore
12.70
自引率
10.40%
发文量
88
审稿时长
62 days
期刊介绍: Vehicular communications is a growing area of communications between vehicles and including roadside communication infrastructure. Advances in wireless communications are making possible sharing of information through real time communications between vehicles and infrastructure. This has led to applications to increase safety of vehicles and communication between passengers and the Internet. Standardization efforts on vehicular communication are also underway to make vehicular transportation safer, greener and easier. The aim of the journal is to publish high quality peer–reviewed papers in the area of vehicular communications. The scope encompasses all types of communications involving vehicles, including vehicle–to–vehicle and vehicle–to–infrastructure. The scope includes (but not limited to) the following topics related to vehicular communications: Vehicle to vehicle and vehicle to infrastructure communications Channel modelling, modulating and coding Congestion Control and scalability issues Protocol design, testing and verification Routing in vehicular networks Security issues and countermeasures Deployment and field testing Reducing energy consumption and enhancing safety of vehicles Wireless in–car networks Data collection and dissemination methods Mobility and handover issues Safety and driver assistance applications UAV Underwater communications Autonomous cooperative driving Social networks Internet of vehicles Standardization of protocols.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信