MAPPO-ITD3-IMLFQ algorithm for multi-mobile robot path planning

IF 8 1区工程技术 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Advanced Engineering Informatics Pub Date : 2025-05-01 DOI:10.1016/j.aei.2025.103398

Likun Hu, Chunyou Wei, Linfei Yin

{"title":"MAPPO-ITD3-IMLFQ algorithm for multi-mobile robot path planning","authors":"Likun Hu, Chunyou Wei, Linfei Yin","doi":"10.1016/j.aei.2025.103398","DOIUrl":null,"url":null,"abstract":"<div><div>With the development of robotics, mobile robots (MRs) are widely applied in industrial and agricultural production. Reasonable path planning (PP) algorithms are the prerequisite for multi-mobile robot (MMR) systems to accomplish tasks. However, the existing PP algorithms of MMR systems still have the problems of being unable to dynamically assign tasks, not comprehensively considering the needs of kinematic constraints and dynamic obstacle avoidance, and poorly coordinating path conflicts. This study proposes a multi-agent proximal policy optimization-artificial potential field twin delayed deep deterministic policy gradient-improved multi-level feedback queue (MAPPO-ITD3-IMLFQ) algorithm for the PP of MMR systems. The proposed MAPPO-ITD3-IMLFQ algorithm combines the multi-agent proximal policy optimization (MAPPO) algorithm, the improved twin delayed deep deterministic policy gradient (ITD3) algorithm, and the improved multi-level feedback queue (IMLFQ) algorithm to form a PP algorithm for MMR system. The MRs apply the MAPPO algorithm to calculate task assignment (TA) schemes and provide sub-goal points for ITD3 algorithm. The MRs apply the ITD3 algorithm to calculate the path of the MRs. When the paths of different MRs conflict, the MR applies the IMLFQ algorithm to coordinate the movement of the MRs. The proposed MAPPO-ITD3-IMLFQ algorithm realizes the dynamic TA of the MMR system, meets the kinematic constraints and dynamic obstacle avoidance requirements of MRs, and coordinates path conflicts among the MRs. In this study, the proposed MAPPO-ITD3-IMLFQ algorithm is applied to different environments for the PP of MMRs. Experimental results show that: compared to the Hungarian algorithm and the genetic algorithm, the proposed MAPPO-ITD3-IMLFQ algorithm reduces the time spent on assigned tasks by 75.25 % and 77.44 %, respectively. Compared to the PP algorithms for reinforcement learning, the proposed MAPPO-ITD3-IMLFQ algorithm reduces the length of the planned path by 23.57 % on average.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"65 ","pages":"Article 103398"},"PeriodicalIF":8.0000,"publicationDate":"2025-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Advanced Engineering Informatics","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1474034625002915","RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}

引用次数: 0

Abstract

With the development of robotics, mobile robots (MRs) are widely applied in industrial and agricultural production. Reasonable path planning (PP) algorithms are the prerequisite for multi-mobile robot (MMR) systems to accomplish tasks. However, the existing PP algorithms of MMR systems still have the problems of being unable to dynamically assign tasks, not comprehensively considering the needs of kinematic constraints and dynamic obstacle avoidance, and poorly coordinating path conflicts. This study proposes a multi-agent proximal policy optimization-artificial potential field twin delayed deep deterministic policy gradient-improved multi-level feedback queue (MAPPO-ITD3-IMLFQ) algorithm for the PP of MMR systems. The proposed MAPPO-ITD3-IMLFQ algorithm combines the multi-agent proximal policy optimization (MAPPO) algorithm, the improved twin delayed deep deterministic policy gradient (ITD3) algorithm, and the improved multi-level feedback queue (IMLFQ) algorithm to form a PP algorithm for MMR system. The MRs apply the MAPPO algorithm to calculate task assignment (TA) schemes and provide sub-goal points for ITD3 algorithm. The MRs apply the ITD3 algorithm to calculate the path of the MRs. When the paths of different MRs conflict, the MR applies the IMLFQ algorithm to coordinate the movement of the MRs. The proposed MAPPO-ITD3-IMLFQ algorithm realizes the dynamic TA of the MMR system, meets the kinematic constraints and dynamic obstacle avoidance requirements of MRs, and coordinates path conflicts among the MRs. In this study, the proposed MAPPO-ITD3-IMLFQ algorithm is applied to different environments for the PP of MMRs. Experimental results show that: compared to the Hungarian algorithm and the genetic algorithm, the proposed MAPPO-ITD3-IMLFQ algorithm reduces the time spent on assigned tasks by 75.25 % and 77.44 %, respectively. Compared to the PP algorithms for reinforcement learning, the proposed MAPPO-ITD3-IMLFQ algorithm reduces the length of the planned path by 23.57 % on average.

查看原文本刊更多论文

多移动机器人路径规划的MAPPO-ITD3-IMLFQ算法

随着机器人技术的发展，移动机器人在工农业生产中得到了广泛的应用。合理的路径规划算法是多移动机器人系统完成任务的前提。然而，现有的MMR系统PP算法仍然存在不能动态分配任务、不能综合考虑运动约束和动态避障需求、路径冲突协调能力差等问题。针对MMR系统的PP问题，提出了一种多智能体近端策略优化-人工势场双延迟深度确定性策略梯度-改进多级反馈队列（MAPPO-ITD3-IMLFQ）算法。本文提出的MAPPO-ITD3-IMLFQ算法结合了多智能体近端策略优化（MAPPO）算法、改进的双延迟深度确定性策略梯度（ITD3）算法和改进的多级反馈队列（IMLFQ）算法，形成了MMR系统的PP算法。MRs采用MAPPO算法计算任务分配方案，并为ITD3算法提供子目标点。MRs采用ITD3算法计算MRs的路径，当不同MRs的路径发生冲突时，MR采用IMLFQ算法协调MRs的运动。本文提出的MAPPO-ITD3-IMLFQ算法实现了MMR系统的动态TA，满足MRs的运动学约束和动态避障要求，并协调MRs之间的路径冲突。将所提出的MAPPO-ITD3-IMLFQ算法应用于mmr的不同PP环境。实验结果表明：与匈牙利算法和遗传算法相比，MAPPO-ITD3-IMLFQ算法分配任务的时间分别减少了75.25%和77.44%。与用于强化学习的PP算法相比，本文提出的MAPPO-ITD3-IMLFQ算法平均减少了23.57%的规划路径长度。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Advanced Engineering Informatics 工程技术-工程：综合

CiteScore

12.40

自引率

18.20%

发文量

292

审稿时长

45 days

期刊介绍： Advanced Engineering Informatics is an international Journal that solicits research papers with an emphasis on 'knowledge' and 'engineering applications'. The Journal seeks original papers that report progress in applying methods of engineering informatics. These papers should have engineering relevance and help provide a scientific base for more reliable, spontaneous, and creative engineering decision-making. Additionally, papers should demonstrate the science of supporting knowledge-intensive engineering tasks and validate the generality, power, and scalability of new methods through rigorous evaluation, preferably both qualitatively and quantitatively. Abstracting and indexing for Advanced Engineering Informatics include Science Citation Index Expanded, Scopus and INSPEC.