{"title":"基于同步积分强化学习的输入饱和时滞非线性多智能体系统最优共识控制","authors":"Chaoyue Li, Fei Wang, Yunliang Wei, Chuan Zhang","doi":"10.1016/j.ejcon.2025.101262","DOIUrl":null,"url":null,"abstract":"<div><div>This paper addresses the optimal consensus control (OCC) problem for nonlinear multi-agent systems (MASs) with input saturation and input delay based on synchronous integral reinforcement learning (IRL). First, the model reduction method is employed to convert the original system into a delay-free model, subsequently, the new performance index functions are introduced, based on which an equivalence relationship is established between the performance indices of two MASs. Through this equivalence, the challenging problem of OCC for MAS with input delays can be successfully transformed into that of delay-free MAS. Second, the Hamilton–Jacobi–Bellman (HJB) equations with non-quadratic functions are established. It is further demonstrated that the solutions to these coupled HJB equations not only are optimal control policies but also constitute Nash equilibrium. Third, the online synchronized IRL algorithm is utilized to design the optimal controllers, constructing actor–critic (A–C) neural networks (NNs) structure to approximate the control policies and value function, respectively. The weights of both NNs are updated synchronously. Finally, the simulation example shows the effectiveness of the method.</div></div>","PeriodicalId":50489,"journal":{"name":"European Journal of Control","volume":"85 ","pages":"Article 101262"},"PeriodicalIF":2.5000,"publicationDate":"2025-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Optimal consensus control for input-delay nonlinear multi-agent systems with input saturation utilizing synchronous integral reinforcement learning\",\"authors\":\"Chaoyue Li, Fei Wang, Yunliang Wei, Chuan Zhang\",\"doi\":\"10.1016/j.ejcon.2025.101262\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><div>This paper addresses the optimal consensus control (OCC) problem for nonlinear multi-agent systems (MASs) with input saturation and input delay based on synchronous integral reinforcement learning (IRL). First, the model reduction method is employed to convert the original system into a delay-free model, subsequently, the new performance index functions are introduced, based on which an equivalence relationship is established between the performance indices of two MASs. Through this equivalence, the challenging problem of OCC for MAS with input delays can be successfully transformed into that of delay-free MAS. Second, the Hamilton–Jacobi–Bellman (HJB) equations with non-quadratic functions are established. It is further demonstrated that the solutions to these coupled HJB equations not only are optimal control policies but also constitute Nash equilibrium. Third, the online synchronized IRL algorithm is utilized to design the optimal controllers, constructing actor–critic (A–C) neural networks (NNs) structure to approximate the control policies and value function, respectively. The weights of both NNs are updated synchronously. Finally, the simulation example shows the effectiveness of the method.</div></div>\",\"PeriodicalId\":50489,\"journal\":{\"name\":\"European Journal of Control\",\"volume\":\"85 \",\"pages\":\"Article 101262\"},\"PeriodicalIF\":2.5000,\"publicationDate\":\"2025-07-11\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"European Journal of Control\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0947358025000913\",\"RegionNum\":3,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"AUTOMATION & CONTROL SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"European Journal of Control","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0947358025000913","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"AUTOMATION & CONTROL SYSTEMS","Score":null,"Total":0}
Optimal consensus control for input-delay nonlinear multi-agent systems with input saturation utilizing synchronous integral reinforcement learning
This paper addresses the optimal consensus control (OCC) problem for nonlinear multi-agent systems (MASs) with input saturation and input delay based on synchronous integral reinforcement learning (IRL). First, the model reduction method is employed to convert the original system into a delay-free model, subsequently, the new performance index functions are introduced, based on which an equivalence relationship is established between the performance indices of two MASs. Through this equivalence, the challenging problem of OCC for MAS with input delays can be successfully transformed into that of delay-free MAS. Second, the Hamilton–Jacobi–Bellman (HJB) equations with non-quadratic functions are established. It is further demonstrated that the solutions to these coupled HJB equations not only are optimal control policies but also constitute Nash equilibrium. Third, the online synchronized IRL algorithm is utilized to design the optimal controllers, constructing actor–critic (A–C) neural networks (NNs) structure to approximate the control policies and value function, respectively. The weights of both NNs are updated synchronously. Finally, the simulation example shows the effectiveness of the method.
期刊介绍:
The European Control Association (EUCA) has among its objectives to promote the development of the discipline. Apart from the European Control Conferences, the European Journal of Control is the Association''s main channel for the dissemination of important contributions in the field.
The aim of the Journal is to publish high quality papers on the theory and practice of control and systems engineering.
The scope of the Journal will be wide and cover all aspects of the discipline including methodologies, techniques and applications.
Research in control and systems engineering is necessary to develop new concepts and tools which enhance our understanding and improve our ability to design and implement high performance control systems. Submitted papers should stress the practical motivations and relevance of their results.
The design and implementation of a successful control system requires the use of a range of techniques:
Modelling
Robustness Analysis
Identification
Optimization
Control Law Design
Numerical analysis
Fault Detection, and so on.