Adaptive Edge Task Offloading via Parameterized Multi-Objective Reinforcement Learning With Hybrid Action Space

IF 7.9 2区计算机科学 Q1 ENGINEERING, MULTIDISCIPLINARY

IEEE Transactions on Network Science and Engineering Pub Date : 2025-06-09 DOI:10.1109/TNSE.2025.3577628

Huimin Tong;Cheng Chen;Weihao Jiang;Ting Wang;Jiang Zhu

{"title":"Adaptive Edge Task Offloading via Parameterized Multi-Objective Reinforcement Learning With Hybrid Action Space","authors":"Huimin Tong;Cheng Chen;Weihao Jiang;Ting Wang;Jiang Zhu","doi":"10.1109/TNSE.2025.3577628","DOIUrl":null,"url":null,"abstract":"In 6G networks, Multi-access Edge Computing (MEC) enables ultra-low latency and high reliability for Internet of Things (IoT) applications. However, optimizing resource allocation in MEC is challenging due to dynamic network conditions and limited computational resources. To address these challenges, this study proposes a Hybrid Multi-Objective Soft Actor-Critic (HMO-SAC) algorithm, which integrates Multi-Objective Reinforcement Learning (MORL) within a hybrid action space. The method dynamically balances multiple optimization objectives, leveraging a hybrid action space to make decisions involving both discrete and continuous parameters, such as task offloading targets and resource allocation. Additionally, an Improved Near-on Experience Replay (INER) mechanism is introduced to mitigate extrapolation errors in off-policy sampled data. Simulation results demonstrate that HMO-SAC improves convergence speed by 14% on average and reduces the task completion time and energy consumption by 23% compared to state-of-the-art methods.","PeriodicalId":54229,"journal":{"name":"IEEE Transactions on Network Science and Engineering","volume":"12 6","pages":"4876-4893"},"PeriodicalIF":7.9000,"publicationDate":"2025-06-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Network Science and Engineering","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/11027806/","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, MULTIDISCIPLINARY","Score":null,"Total":0}

引用次数: 0

Abstract

In 6G networks, Multi-access Edge Computing (MEC) enables ultra-low latency and high reliability for Internet of Things (IoT) applications. However, optimizing resource allocation in MEC is challenging due to dynamic network conditions and limited computational resources. To address these challenges, this study proposes a Hybrid Multi-Objective Soft Actor-Critic (HMO-SAC) algorithm, which integrates Multi-Objective Reinforcement Learning (MORL) within a hybrid action space. The method dynamically balances multiple optimization objectives, leveraging a hybrid action space to make decisions involving both discrete and continuous parameters, such as task offloading targets and resource allocation. Additionally, an Improved Near-on Experience Replay (INER) mechanism is introduced to mitigate extrapolation errors in off-policy sampled data. Simulation results demonstrate that HMO-SAC improves convergence speed by 14% on average and reduces the task completion time and energy consumption by 23% compared to state-of-the-art methods.

查看原文本刊更多论文

基于混合动作空间的参数化多目标强化学习自适应边缘任务卸载

在6G网络中，多接入边缘计算（MEC）可为物联网（IoT）应用提供超低延迟和高可靠性。然而，由于网络环境的动态性和计算资源的有限性，优化MEC中的资源分配具有一定的挑战性。为了解决这些挑战，本研究提出了一种混合多目标软行为者-评论家（HMO-SAC）算法，该算法在混合动作空间中集成了多目标强化学习（MORL）。该方法动态平衡多个优化目标，利用混合动作空间来做出涉及离散和连续参数的决策，如任务卸载目标和资源分配。此外，还引入了改进的近距离体验重放（INER）机制，以减轻非策略采样数据的外推误差。仿真结果表明，与现有方法相比，HMO-SAC算法的收敛速度平均提高14%，任务完成时间和能耗平均降低23%。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

IEEE Transactions on Network Science and Engineering Engineering-Control and Systems Engineering

CiteScore

12.60

自引率

9.10%

发文量

393

期刊介绍： The proposed journal, called the IEEE Transactions on Network Science and Engineering (TNSE), is committed to timely publishing of peer-reviewed technical articles that deal with the theory and applications of network science and the interconnections among the elements in a system that form a network. In particular, the IEEE Transactions on Network Science and Engineering publishes articles on understanding, prediction, and control of structures and behaviors of networks at the fundamental level. The types of networks covered include physical or engineered networks, information networks, biological networks, semantic networks, economic networks, social networks, and ecological networks. Aimed at discovering common principles that govern network structures, network functionalities and behaviors of networks, the journal seeks articles on understanding, prediction, and control of structures and behaviors of networks. Another trans-disciplinary focus of the IEEE Transactions on Network Science and Engineering is the interactions between and co-evolution of different genres of networks.