Distributed optimal control of nonlinear multi‐agent systems based on integral reinforcement learning

Ying Xu, Kewen Li, Yongming Li
{"title":"Distributed optimal control of nonlinear multi‐agent systems based on integral reinforcement learning","authors":"Ying Xu, Kewen Li, Yongming Li","doi":"10.1002/oca.3174","DOIUrl":null,"url":null,"abstract":"In this article, a distributed optimal control approach is proposed for a class of affine nonlinear multi‐agent systems (MASs) with unknown nonlinear dynamics. The game theory is used to formulate the distributed optimal control problem into a differential graphical game problem with synchronized updates of all nodes. A data‐based integral reinforcement learning (IRL) algorithm is used to learn the solution of the coupled Hamilton–Jacobi (HJ) equation without prior knowledge of the drift dynamics, and the actor‐critic neural networks (A‐C NNs) are used to approximate the control law and the cost function, respectively. To update the parameters synchronously, the gradient descent algorithm is used to design the weight update laws of the A‐C NNs. Combining the IRL and the A‐C NNs, a distributed consensus optimal control method is designed. By using the Lyapunov stability theory, the developed optimal control method can show that all signals in the considered system are uniformly ultimately bounded (UUB), and the systems can achieve Nash equilibrium when all agents update their controllers simultaneously. Finally, simulation results are given to illustrate the effectiveness of the developed optimal control approach.","PeriodicalId":501055,"journal":{"name":"Optimal Control Applications and Methods","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2024-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Optimal Control Applications and Methods","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1002/oca.3174","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

Abstract

In this article, a distributed optimal control approach is proposed for a class of affine nonlinear multi‐agent systems (MASs) with unknown nonlinear dynamics. The game theory is used to formulate the distributed optimal control problem into a differential graphical game problem with synchronized updates of all nodes. A data‐based integral reinforcement learning (IRL) algorithm is used to learn the solution of the coupled Hamilton–Jacobi (HJ) equation without prior knowledge of the drift dynamics, and the actor‐critic neural networks (A‐C NNs) are used to approximate the control law and the cost function, respectively. To update the parameters synchronously, the gradient descent algorithm is used to design the weight update laws of the A‐C NNs. Combining the IRL and the A‐C NNs, a distributed consensus optimal control method is designed. By using the Lyapunov stability theory, the developed optimal control method can show that all signals in the considered system are uniformly ultimately bounded (UUB), and the systems can achieve Nash equilibrium when all agents update their controllers simultaneously. Finally, simulation results are given to illustrate the effectiveness of the developed optimal control approach.
基于积分强化学习的非线性多代理系统分布式优化控制
本文针对一类具有未知非线性动力学特性的仿射非线性多代理系统(MAS),提出了一种分布式最优控制方法。博弈论被用来将分布式最优控制问题表述为一个所有节点同步更新的微分图形博弈问题。使用基于数据的积分强化学习(IRL)算法来学习耦合汉密尔顿-雅可比(HJ)方程的解,而无需事先了解漂移动力学,并使用行为批判神经网络(A-C NN)分别逼近控制法则和成本函数。为了同步更新参数,采用梯度下降算法设计 A-C 神经网络的权值更新规律。结合 IRL 和 A-C NN,设计了一种分布式共识最优控制方法。通过使用 Lyapunov 稳定性理论,所开发的最优控制方法可以证明所考虑系统中的所有信号都是均匀最终有界的(UUB),并且当所有代理同时更新其控制器时,系统可以达到纳什均衡。最后,还给出了仿真结果,以说明所开发的最优控制方法的有效性。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信