Kai Xue, Linbo Zhai, Yumei Li, Zekun Lu, Wenjie Zhou
{"title":"Task offloading and multi-cache placement based on DRL in UAV-assisted MEC networks","authors":"Kai Xue, Linbo Zhai, Yumei Li, Zekun Lu, Wenjie Zhou","doi":"10.1016/j.vehcom.2025.100900","DOIUrl":null,"url":null,"abstract":"<div><div>Unmanned aerial vehicles (UAVs) are being developed as a promising technology to assist mobile edge computing (MEC) systems due to their reliable wireless communication, flexible computing service capabilities, and flexible deployment. However, in the face of huge information and demanding task delay, it is a challenging problem to reduce the system cost. This paper studies task offloading and cache space placement for ground users, and proposes a multi-UAV assisted computing framework, which is a four-layer transmission system composed of ground users (UE), UAVs, edge data centers (EDC) and remote clouds. By jointly optimizing UAV cache space, flight path, offloading decision, channel ratio, and battery power, we formulate the problem to minimize the long-term average weighted cost of the system under the constraint of cache space and computing resources. Since this problem is a mixed integer variable problem, we design a task offloading and cache placement algorithm based on deep reinforcement learning, namely the Cooperative Long-term Average Cost Minimization Optimization Algorithm (CLACMO). Firstly, we transform the mixed action variable space by using embedded tables and conditional variational autoencoders (VAE) combined with latent space, and map the mixed action variable to the latent action space. This approach effectively unifies discrete and continuous actions, addressing the challenge of mixed action spaces that traditional deep reinforcement learning algorithms struggle with. Secondly, based on the deep reinforcement learning (DRL), we achieve the long-term system average weighted cost minimization more efficiently under the constraints of task offloading and cache placement. The results show that compared with the PER-UOS-RL, MASAC, and MADDPG algorithms, the average reward has increased by 54.5%, 66.7%, and 69.7% respectively, and the average task completion rate has increased by 12.9%, 38.1%, and 9.11% respectively, demonstrating the effectiveness of our novel method.</div></div>","PeriodicalId":54346,"journal":{"name":"Vehicular Communications","volume":"53 ","pages":"Article 100900"},"PeriodicalIF":5.8000,"publicationDate":"2025-02-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Vehicular Communications","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2214209625000270","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"TELECOMMUNICATIONS","Score":null,"Total":0}
引用次数: 0
Abstract
Unmanned aerial vehicles (UAVs) are being developed as a promising technology to assist mobile edge computing (MEC) systems due to their reliable wireless communication, flexible computing service capabilities, and flexible deployment. However, in the face of huge information and demanding task delay, it is a challenging problem to reduce the system cost. This paper studies task offloading and cache space placement for ground users, and proposes a multi-UAV assisted computing framework, which is a four-layer transmission system composed of ground users (UE), UAVs, edge data centers (EDC) and remote clouds. By jointly optimizing UAV cache space, flight path, offloading decision, channel ratio, and battery power, we formulate the problem to minimize the long-term average weighted cost of the system under the constraint of cache space and computing resources. Since this problem is a mixed integer variable problem, we design a task offloading and cache placement algorithm based on deep reinforcement learning, namely the Cooperative Long-term Average Cost Minimization Optimization Algorithm (CLACMO). Firstly, we transform the mixed action variable space by using embedded tables and conditional variational autoencoders (VAE) combined with latent space, and map the mixed action variable to the latent action space. This approach effectively unifies discrete and continuous actions, addressing the challenge of mixed action spaces that traditional deep reinforcement learning algorithms struggle with. Secondly, based on the deep reinforcement learning (DRL), we achieve the long-term system average weighted cost minimization more efficiently under the constraints of task offloading and cache placement. The results show that compared with the PER-UOS-RL, MASAC, and MADDPG algorithms, the average reward has increased by 54.5%, 66.7%, and 69.7% respectively, and the average task completion rate has increased by 12.9%, 38.1%, and 9.11% respectively, demonstrating the effectiveness of our novel method.
期刊介绍:
Vehicular communications is a growing area of communications between vehicles and including roadside communication infrastructure. Advances in wireless communications are making possible sharing of information through real time communications between vehicles and infrastructure. This has led to applications to increase safety of vehicles and communication between passengers and the Internet. Standardization efforts on vehicular communication are also underway to make vehicular transportation safer, greener and easier.
The aim of the journal is to publish high quality peer–reviewed papers in the area of vehicular communications. The scope encompasses all types of communications involving vehicles, including vehicle–to–vehicle and vehicle–to–infrastructure. The scope includes (but not limited to) the following topics related to vehicular communications:
Vehicle to vehicle and vehicle to infrastructure communications
Channel modelling, modulating and coding
Congestion Control and scalability issues
Protocol design, testing and verification
Routing in vehicular networks
Security issues and countermeasures
Deployment and field testing
Reducing energy consumption and enhancing safety of vehicles
Wireless in–car networks
Data collection and dissemination methods
Mobility and handover issues
Safety and driver assistance applications
UAV
Underwater communications
Autonomous cooperative driving
Social networks
Internet of vehicles
Standardization of protocols.