基于自适应因子评价的无漂移非线性系统最优调节

Ashwin P. Dani;Shubhendu Bhasin
{"title":"基于自适应因子评价的无漂移非线性系统最优调节","authors":"Ashwin P. Dani;Shubhendu Bhasin","doi":"10.1109/OJCSYS.2025.3552999","DOIUrl":null,"url":null,"abstract":"In this paper, a continuous-time adaptive actor-critic reinforcement learning (RL) controller is developed for drift-free uncertain nonlinear systems. Practical examples of such systems are image-based visual servoing (IBVS) and wheeled mobile robots (WMR), where the system dynamics include a parametric uncertainty in the control effectiveness matrix with no drift term. The uncertainty in the input term poses a challenge when developing a continuous-time RL controller using existing methods. This paper presents an actor-critic/synchronous policy iteration (PI)-based RL controller with a newly derived constrained concurrent learning (CCL)-based parameter update law for estimating the unknown parameters of the linearly parametrized control effectiveness matrix. The parameter update law ensures that the parameters do not converge to <inline-formula><tex-math>$zero$</tex-math></inline-formula>, avoiding possible loss of stabilization. An infinite-horizon value function minimization objective is achieved by regulating the current states to the desired with near-optimal control efforts. The proposed controller guarantees closed-loop stability, and simulation results in the presence of noise validate the proposed theory using IBVS and WMR examples.","PeriodicalId":73299,"journal":{"name":"IEEE open journal of control systems","volume":"4 ","pages":"117-129"},"PeriodicalIF":0.0000,"publicationDate":"2025-03-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10932715","citationCount":"0","resultStr":"{\"title\":\"Adaptive Actor-Critic Based Optimal Regulation for Drift-Free Nonlinear Systems\",\"authors\":\"Ashwin P. Dani;Shubhendu Bhasin\",\"doi\":\"10.1109/OJCSYS.2025.3552999\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"In this paper, a continuous-time adaptive actor-critic reinforcement learning (RL) controller is developed for drift-free uncertain nonlinear systems. Practical examples of such systems are image-based visual servoing (IBVS) and wheeled mobile robots (WMR), where the system dynamics include a parametric uncertainty in the control effectiveness matrix with no drift term. The uncertainty in the input term poses a challenge when developing a continuous-time RL controller using existing methods. This paper presents an actor-critic/synchronous policy iteration (PI)-based RL controller with a newly derived constrained concurrent learning (CCL)-based parameter update law for estimating the unknown parameters of the linearly parametrized control effectiveness matrix. The parameter update law ensures that the parameters do not converge to <inline-formula><tex-math>$zero$</tex-math></inline-formula>, avoiding possible loss of stabilization. An infinite-horizon value function minimization objective is achieved by regulating the current states to the desired with near-optimal control efforts. The proposed controller guarantees closed-loop stability, and simulation results in the presence of noise validate the proposed theory using IBVS and WMR examples.\",\"PeriodicalId\":73299,\"journal\":{\"name\":\"IEEE open journal of control systems\",\"volume\":\"4 \",\"pages\":\"117-129\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2025-03-18\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10932715\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE open journal of control systems\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/10932715/\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE open journal of control systems","FirstCategoryId":"1085","ListUrlMain":"https://ieeexplore.ieee.org/document/10932715/","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

摘要

针对无漂移不确定非线性系统,提出了一种连续时间自适应行为者评价强化学习(RL)控制器。此类系统的实际示例是基于图像的视觉伺服(IBVS)和轮式移动机器人(WMR),其中系统动力学包括控制有效性矩阵中没有漂移项的参数不确定性。在使用现有方法开发连续时间RL控制器时,输入项的不确定性提出了挑战。本文提出了一种基于actor-critic/synchronous policy iteration (PI)的RL控制器,该控制器采用了一种新的基于约束并发学习(CCL)的参数更新律来估计线性参数化控制有效性矩阵的未知参数。参数更新律保证了参数不收敛于零,避免了可能的镇定损失。利用接近最优的控制努力将当前状态调节到理想状态,从而实现了无限视界值函数的最小化目标。所提出的控制器保证了闭环稳定性,并且在存在噪声的情况下,使用IBVS和WMR实例的仿真结果验证了所提出的理论。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Adaptive Actor-Critic Based Optimal Regulation for Drift-Free Nonlinear Systems
In this paper, a continuous-time adaptive actor-critic reinforcement learning (RL) controller is developed for drift-free uncertain nonlinear systems. Practical examples of such systems are image-based visual servoing (IBVS) and wheeled mobile robots (WMR), where the system dynamics include a parametric uncertainty in the control effectiveness matrix with no drift term. The uncertainty in the input term poses a challenge when developing a continuous-time RL controller using existing methods. This paper presents an actor-critic/synchronous policy iteration (PI)-based RL controller with a newly derived constrained concurrent learning (CCL)-based parameter update law for estimating the unknown parameters of the linearly parametrized control effectiveness matrix. The parameter update law ensures that the parameters do not converge to $zero$, avoiding possible loss of stabilization. An infinite-horizon value function minimization objective is achieved by regulating the current states to the desired with near-optimal control efforts. The proposed controller guarantees closed-loop stability, and simulation results in the presence of noise validate the proposed theory using IBVS and WMR examples.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信