Reinforcement Learning Control for a Class of Discrete-Time Non-Strict Feedback Multi-Agent Systems and Application to Multi-Marine Vehicles

IF 14.3 1区工程技术 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

IEEE Transactions on Intelligent Vehicles Pub Date : 2024-09-11 DOI:10.1109/TIV.2024.3458894

Weiwei Bai;Dewang Chen;Bo Zhao;Andrea D'Ariano

{"title":"Reinforcement Learning Control for a Class of Discrete-Time Non-Strict Feedback Multi-Agent Systems and Application to Multi-Marine Vehicles","authors":"Weiwei Bai;Dewang Chen;Bo Zhao;Andrea D'Ariano","doi":"10.1109/TIV.2024.3458894","DOIUrl":null,"url":null,"abstract":"A novel control design problem for a class of non-strict feedback multi-agent systems (MAS) in discrete-time form is studied based on reinforcement learning (RL) and applied to multi-marine vehicles (MMV). Firstly, for this kind of discrete-time MAS, a novel system transformation, which can not only solve the noncausal problem that exists in the backstepping method but also reduce the computational complexity, is proposed. Secondly, the algebraic-loop problem inherent in the conventional controller design is solved by compensating the dynamics and using the property of neural network (NN). Thirdly, the multi-gradient recursive (MGR) RL scheme is developed for the sake of designing the optimal controller. Finally, the stability analysis is presented, and all signals are ensured to be semi-global uniformly ultimately bounded (SGUUB) in the Lyapunov's sense. Besides, this scheme is applied to the MMV which can be described in the non-strict feedback form to extend the application of the designed controller. The MMV simulation demonstrates the validation of this scheme.","PeriodicalId":36532,"journal":{"name":"IEEE Transactions on Intelligent Vehicles","volume":"10 5","pages":"3613-3625"},"PeriodicalIF":14.3000,"publicationDate":"2024-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Intelligent Vehicles","FirstCategoryId":"5","ListUrlMain":"https://ieeexplore.ieee.org/document/10678848/","RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}

引用次数: 0

Abstract

A novel control design problem for a class of non-strict feedback multi-agent systems (MAS) in discrete-time form is studied based on reinforcement learning (RL) and applied to multi-marine vehicles (MMV). Firstly, for this kind of discrete-time MAS, a novel system transformation, which can not only solve the noncausal problem that exists in the backstepping method but also reduce the computational complexity, is proposed. Secondly, the algebraic-loop problem inherent in the conventional controller design is solved by compensating the dynamics and using the property of neural network (NN). Thirdly, the multi-gradient recursive (MGR) RL scheme is developed for the sake of designing the optimal controller. Finally, the stability analysis is presented, and all signals are ensured to be semi-global uniformly ultimately bounded (SGUUB) in the Lyapunov's sense. Besides, this scheme is applied to the MMV which can be described in the non-strict feedback form to extend the application of the designed controller. The MMV simulation demonstrates the validation of this scheme.

查看原文本刊更多论文

一类离散时间非严格反馈多智能体系统的强化学习控制及其在多船舶上的应用

研究了基于强化学习（RL）的非严格反馈离散型多智能体系统（MAS）的控制设计问题，并将其应用于多船车辆（MMV）。首先，针对这类离散时间MAS，提出了一种新的系统变换，既能解决退步法存在的非因果问题，又能降低计算复杂度；其次，利用神经网络的特性，利用动态补偿方法解决了传统控制器设计中固有的代数环问题；第三，为了设计最优控制器，提出了多梯度递推RL方案。最后进行了稳定性分析，保证了所有信号在Lyapunov意义下是半全局一致最终有界的。此外，将该方案应用于可以用非严格反馈形式描述的MMV，扩展了所设计控制器的应用范围。通过MMV仿真验证了该方案的有效性。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

IEEE Transactions on Intelligent Vehicles Mathematics-Control and Optimization

CiteScore

12.10

自引率

13.40%

发文量

177

期刊介绍： The IEEE Transactions on Intelligent Vehicles (T-IV) is a premier platform for publishing peer-reviewed articles that present innovative research concepts, application results, significant theoretical findings, and application case studies in the field of intelligent vehicles. With a particular emphasis on automated vehicles within roadway environments, T-IV aims to raise awareness of pressing research and application challenges. Our focus is on providing critical information to the intelligent vehicle community, serving as a dissemination vehicle for IEEE ITS Society members and others interested in learning about the state-of-the-art developments and progress in research and applications related to intelligent vehicles. Join us in advancing knowledge and innovation in this dynamic field.