{"title":"基于混合动作空间的参数化多目标强化学习自适应边缘任务卸载","authors":"Huimin Tong;Cheng Chen;Weihao Jiang;Ting Wang;Jiang Zhu","doi":"10.1109/TNSE.2025.3577628","DOIUrl":null,"url":null,"abstract":"In 6G networks, Multi-access Edge Computing (MEC) enables ultra-low latency and high reliability for Internet of Things (IoT) applications. However, optimizing resource allocation in MEC is challenging due to dynamic network conditions and limited computational resources. To address these challenges, this study proposes a Hybrid Multi-Objective Soft Actor-Critic (HMO-SAC) algorithm, which integrates Multi-Objective Reinforcement Learning (MORL) within a hybrid action space. The method dynamically balances multiple optimization objectives, leveraging a hybrid action space to make decisions involving both discrete and continuous parameters, such as task offloading targets and resource allocation. Additionally, an Improved Near-on Experience Replay (INER) mechanism is introduced to mitigate extrapolation errors in off-policy sampled data. Simulation results demonstrate that HMO-SAC improves convergence speed by 14% on average and reduces the task completion time and energy consumption by 23% compared to state-of-the-art methods.","PeriodicalId":54229,"journal":{"name":"IEEE Transactions on Network Science and Engineering","volume":"12 6","pages":"4876-4893"},"PeriodicalIF":7.9000,"publicationDate":"2025-06-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Adaptive Edge Task Offloading via Parameterized Multi-Objective Reinforcement Learning With Hybrid Action Space\",\"authors\":\"Huimin Tong;Cheng Chen;Weihao Jiang;Ting Wang;Jiang Zhu\",\"doi\":\"10.1109/TNSE.2025.3577628\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"In 6G networks, Multi-access Edge Computing (MEC) enables ultra-low latency and high reliability for Internet of Things (IoT) applications. However, optimizing resource allocation in MEC is challenging due to dynamic network conditions and limited computational resources. To address these challenges, this study proposes a Hybrid Multi-Objective Soft Actor-Critic (HMO-SAC) algorithm, which integrates Multi-Objective Reinforcement Learning (MORL) within a hybrid action space. The method dynamically balances multiple optimization objectives, leveraging a hybrid action space to make decisions involving both discrete and continuous parameters, such as task offloading targets and resource allocation. Additionally, an Improved Near-on Experience Replay (INER) mechanism is introduced to mitigate extrapolation errors in off-policy sampled data. Simulation results demonstrate that HMO-SAC improves convergence speed by 14% on average and reduces the task completion time and energy consumption by 23% compared to state-of-the-art methods.\",\"PeriodicalId\":54229,\"journal\":{\"name\":\"IEEE Transactions on Network Science and Engineering\",\"volume\":\"12 6\",\"pages\":\"4876-4893\"},\"PeriodicalIF\":7.9000,\"publicationDate\":\"2025-06-09\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE Transactions on Network Science and Engineering\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/11027806/\",\"RegionNum\":2,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"ENGINEERING, MULTIDISCIPLINARY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Network Science and Engineering","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/11027806/","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, MULTIDISCIPLINARY","Score":null,"Total":0}
Adaptive Edge Task Offloading via Parameterized Multi-Objective Reinforcement Learning With Hybrid Action Space
In 6G networks, Multi-access Edge Computing (MEC) enables ultra-low latency and high reliability for Internet of Things (IoT) applications. However, optimizing resource allocation in MEC is challenging due to dynamic network conditions and limited computational resources. To address these challenges, this study proposes a Hybrid Multi-Objective Soft Actor-Critic (HMO-SAC) algorithm, which integrates Multi-Objective Reinforcement Learning (MORL) within a hybrid action space. The method dynamically balances multiple optimization objectives, leveraging a hybrid action space to make decisions involving both discrete and continuous parameters, such as task offloading targets and resource allocation. Additionally, an Improved Near-on Experience Replay (INER) mechanism is introduced to mitigate extrapolation errors in off-policy sampled data. Simulation results demonstrate that HMO-SAC improves convergence speed by 14% on average and reduces the task completion time and energy consumption by 23% compared to state-of-the-art methods.
期刊介绍:
The proposed journal, called the IEEE Transactions on Network Science and Engineering (TNSE), is committed to timely publishing of peer-reviewed technical articles that deal with the theory and applications of network science and the interconnections among the elements in a system that form a network. In particular, the IEEE Transactions on Network Science and Engineering publishes articles on understanding, prediction, and control of structures and behaviors of networks at the fundamental level. The types of networks covered include physical or engineered networks, information networks, biological networks, semantic networks, economic networks, social networks, and ecological networks. Aimed at discovering common principles that govern network structures, network functionalities and behaviors of networks, the journal seeks articles on understanding, prediction, and control of structures and behaviors of networks. Another trans-disciplinary focus of the IEEE Transactions on Network Science and Engineering is the interactions between and co-evolution of different genres of networks.