面向工业物联网任务卸载的进化多目标深度强化学习

IF 5.4 2区 计算机科学 Q1 COMPUTER SCIENCE, INFORMATION SYSTEMS
Xu Liu;Zheng-Yi Chai;Yan-Yang Cheng;Ya-Lun Li;Tao Li
{"title":"面向工业物联网任务卸载的进化多目标深度强化学习","authors":"Xu Liu;Zheng-Yi Chai;Yan-Yang Cheng;Ya-Lun Li;Tao Li","doi":"10.1109/TNSM.2025.3585148","DOIUrl":null,"url":null,"abstract":"Mobile Edge Computing (MEC) plays a pivotal role in optimizing the Industrial Internet of Things (IIoT), where the Industrial Task Offloading Problem (ITOP) is crucial for ensuring optimal system performance by balancing conflicting objectives such as delay, energy consumption, and cost. However, existing approaches often oversimplify multi-objective optimization by aggregating conflicting goals into a single objective, while also suffering from limited exploration and robustness in uncertain MEC scenarios within IIoT. To overcome this limitation, we propose EMDRL-ITOP, an Evolutionary Multi-Objective Deep Reinforcement Learning algorithm that synergizes evolutionary algorithm with deep reinforcement learning (DRL). Firstly, we formulate a multi-objective task scheduling model for IIoT-MEC and design a three-dimensional vector reward function within a Multi-Objective Markov Decision Process framework, enabling simultaneous optimization of delay, energy, and cost. Then, EMDRL-ITOP integrates evolutionary mechanisms to enhance exploration and robustness: a dynamic elite selection strategy prioritizes high-quality policies, a distillation crossover operator fuses advantageous traits from elite strategies, and a proximal mutation mechanism maintains population diversity. These components collectively improve learning efficiency and solution quality in dynamic environments. Extensive simulations across six instances demonstrate that EMDRL-ITOP achieves a superior balance among conflicting objectives compared to state-of-the-art methods, while also outperforming existing algorithms in several key performance metrics.","PeriodicalId":13423,"journal":{"name":"IEEE Transactions on Network and Service Management","volume":"22 5","pages":"5074-5089"},"PeriodicalIF":5.4000,"publicationDate":"2025-07-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Evolutionary Multi-Objective Deep Reinforcement Learning for Task Offloading in Industrial Internet of Things\",\"authors\":\"Xu Liu;Zheng-Yi Chai;Yan-Yang Cheng;Ya-Lun Li;Tao Li\",\"doi\":\"10.1109/TNSM.2025.3585148\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Mobile Edge Computing (MEC) plays a pivotal role in optimizing the Industrial Internet of Things (IIoT), where the Industrial Task Offloading Problem (ITOP) is crucial for ensuring optimal system performance by balancing conflicting objectives such as delay, energy consumption, and cost. However, existing approaches often oversimplify multi-objective optimization by aggregating conflicting goals into a single objective, while also suffering from limited exploration and robustness in uncertain MEC scenarios within IIoT. To overcome this limitation, we propose EMDRL-ITOP, an Evolutionary Multi-Objective Deep Reinforcement Learning algorithm that synergizes evolutionary algorithm with deep reinforcement learning (DRL). Firstly, we formulate a multi-objective task scheduling model for IIoT-MEC and design a three-dimensional vector reward function within a Multi-Objective Markov Decision Process framework, enabling simultaneous optimization of delay, energy, and cost. Then, EMDRL-ITOP integrates evolutionary mechanisms to enhance exploration and robustness: a dynamic elite selection strategy prioritizes high-quality policies, a distillation crossover operator fuses advantageous traits from elite strategies, and a proximal mutation mechanism maintains population diversity. These components collectively improve learning efficiency and solution quality in dynamic environments. Extensive simulations across six instances demonstrate that EMDRL-ITOP achieves a superior balance among conflicting objectives compared to state-of-the-art methods, while also outperforming existing algorithms in several key performance metrics.\",\"PeriodicalId\":13423,\"journal\":{\"name\":\"IEEE Transactions on Network and Service Management\",\"volume\":\"22 5\",\"pages\":\"5074-5089\"},\"PeriodicalIF\":5.4000,\"publicationDate\":\"2025-07-02\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE Transactions on Network and Service Management\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/11063458/\",\"RegionNum\":2,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"COMPUTER SCIENCE, INFORMATION SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Network and Service Management","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/11063458/","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0

摘要

移动边缘计算(MEC)在优化工业物联网(IIoT)中发挥着关键作用,其中工业任务卸载问题(ITOP)对于通过平衡延迟、能耗和成本等相互冲突的目标来确保最佳系统性能至关重要。然而,现有的方法往往通过将相互冲突的目标聚合到单个目标中来过度简化多目标优化,同时在工业物联网中不确定的MEC场景中,探索和鲁棒性也有限。为了克服这一限制,我们提出了EMDRL-ITOP,一种将进化算法与深度强化学习(DRL)相结合的进化多目标深度强化学习算法。首先,我们建立了IIoT-MEC的多目标任务调度模型,并在多目标马尔可夫决策过程框架内设计了三维矢量奖励函数,实现了延迟、能量和成本的同时优化。然后,EMDRL-ITOP整合了进化机制,以增强探索和鲁棒性:动态精英选择策略优先选择优质策略,蒸馏交叉算子融合精英策略中的优势性状,近端突变机制保持种群多样性。这些组件共同提高了动态环境中的学习效率和解决方案质量。六个实例的广泛模拟表明,与最先进的方法相比,EMDRL-ITOP在冲突目标之间实现了更好的平衡,同时在几个关键性能指标上也优于现有算法。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Evolutionary Multi-Objective Deep Reinforcement Learning for Task Offloading in Industrial Internet of Things
Mobile Edge Computing (MEC) plays a pivotal role in optimizing the Industrial Internet of Things (IIoT), where the Industrial Task Offloading Problem (ITOP) is crucial for ensuring optimal system performance by balancing conflicting objectives such as delay, energy consumption, and cost. However, existing approaches often oversimplify multi-objective optimization by aggregating conflicting goals into a single objective, while also suffering from limited exploration and robustness in uncertain MEC scenarios within IIoT. To overcome this limitation, we propose EMDRL-ITOP, an Evolutionary Multi-Objective Deep Reinforcement Learning algorithm that synergizes evolutionary algorithm with deep reinforcement learning (DRL). Firstly, we formulate a multi-objective task scheduling model for IIoT-MEC and design a three-dimensional vector reward function within a Multi-Objective Markov Decision Process framework, enabling simultaneous optimization of delay, energy, and cost. Then, EMDRL-ITOP integrates evolutionary mechanisms to enhance exploration and robustness: a dynamic elite selection strategy prioritizes high-quality policies, a distillation crossover operator fuses advantageous traits from elite strategies, and a proximal mutation mechanism maintains population diversity. These components collectively improve learning efficiency and solution quality in dynamic environments. Extensive simulations across six instances demonstrate that EMDRL-ITOP achieves a superior balance among conflicting objectives compared to state-of-the-art methods, while also outperforming existing algorithms in several key performance metrics.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
IEEE Transactions on Network and Service Management
IEEE Transactions on Network and Service Management Computer Science-Computer Networks and Communications
CiteScore
9.30
自引率
15.10%
发文量
325
期刊介绍: IEEE Transactions on Network and Service Management will publish (online only) peerreviewed archival quality papers that advance the state-of-the-art and practical applications of network and service management. Theoretical research contributions (presenting new concepts and techniques) and applied contributions (reporting on experiences and experiments with actual systems) will be encouraged. These transactions will focus on the key technical issues related to: Management Models, Architectures and Frameworks; Service Provisioning, Reliability and Quality Assurance; Management Functions; Enabling Technologies; Information and Communication Models; Policies; Applications and Case Studies; Emerging Technologies and Standards.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信