Swarm-intelligence-based value iteration for optimal regulation of continuous-time nonlinear systems

IF 8.2 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Swarm and Evolutionary Computation Pub Date : 2025-03-22 DOI:10.1016/j.swevo.2025.101913

Ding Wang, Qinna Hu, Ao Liu, Junfei Qiao

{"title":"Swarm-intelligence-based value iteration for optimal regulation of continuous-time nonlinear systems","authors":"Ding Wang, Qinna Hu, Ao Liu, Junfei Qiao","doi":"10.1016/j.swevo.2025.101913","DOIUrl":null,"url":null,"abstract":"<div><div>In this article, a swarm-intelligence-based value iteration (VI) algorithm is constructed to resolve the optimal control issue for continuous-time (CT) nonlinear systems. By leveraging the evolutionary concept of particle swarm optimization (PSO), the challenge of gradient vanishing is effectively overcome compared to traditional adaptive dynamic programming (ADP). Specifically, a PSO-based action network is implemented to perform policy improvement, eliminating the reliance on gradient information. Furthermore, within the ADP framework, the swarm-intelligence-based VI algorithm for CT systems is developed to address the challenges associated with constraints of initial admissible conditions and the difficulty of selecting probing signals in the traditional policy iteration method. The theoretical analysis is provided to show the convergence of the developed VI algorithm and the stability of the closed-loop system, respectively. Finally, under affine and non-affine backgrounds, two simulations are conducted to demonstrate the effectiveness and optimality of the established swarm-intelligence-based VI scheme for CT systems.</div></div>","PeriodicalId":48682,"journal":{"name":"Swarm and Evolutionary Computation","volume":"95 ","pages":"Article 101913"},"PeriodicalIF":8.2000,"publicationDate":"2025-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Swarm and Evolutionary Computation","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2210650225000719","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}

引用次数: 0

Abstract

In this article, a swarm-intelligence-based value iteration (VI) algorithm is constructed to resolve the optimal control issue for continuous-time (CT) nonlinear systems. By leveraging the evolutionary concept of particle swarm optimization (PSO), the challenge of gradient vanishing is effectively overcome compared to traditional adaptive dynamic programming (ADP). Specifically, a PSO-based action network is implemented to perform policy improvement, eliminating the reliance on gradient information. Furthermore, within the ADP framework, the swarm-intelligence-based VI algorithm for CT systems is developed to address the challenges associated with constraints of initial admissible conditions and the difficulty of selecting probing signals in the traditional policy iteration method. The theoretical analysis is provided to show the convergence of the developed VI algorithm and the stability of the closed-loop system, respectively. Finally, under affine and non-affine backgrounds, two simulations are conducted to demonstrate the effectiveness and optimality of the established swarm-intelligence-based VI scheme for CT systems.

查看原文本刊更多论文

基于群智能的连续非线性系统最优调节值迭代

针对连续时间非线性系统的最优控制问题，提出了一种基于群体智能的值迭代算法。与传统的自适应动态规划（ADP）相比，利用粒子群优化（PSO）的进化概念，有效地克服了梯度消失的挑战。具体来说，实现了基于pso的行动网络来执行策略改进，消除了对梯度信息的依赖。此外，在ADP框架下，开发了基于群体智能的CT系统VI算法，解决了传统策略迭代方法中初始允许条件约束和探测信号选择困难的问题。理论分析分别证明了所开发的VI算法的收敛性和闭环系统的稳定性。最后，在仿射和非仿射背景下，进行了两次仿真，验证了所建立的基于群体智能的CT系统VI方案的有效性和最优性。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Swarm and Evolutionary Computation COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCEC-COMPUTER SCIENCE, THEORY & METHODS

CiteScore

16.00

自引率

12.00%

发文量

169

期刊介绍： Swarm and Evolutionary Computation is a pioneering peer-reviewed journal focused on the latest research and advancements in nature-inspired intelligent computation using swarm and evolutionary algorithms. It covers theoretical, experimental, and practical aspects of these paradigms and their hybrids, promoting interdisciplinary research. The journal prioritizes the publication of high-quality, original articles that push the boundaries of evolutionary computation and swarm intelligence. Additionally, it welcomes survey papers on current topics and novel applications. Topics of interest include but are not limited to: Genetic Algorithms, and Genetic Programming, Evolution Strategies, and Evolutionary Programming, Differential Evolution, Artificial Immune Systems, Particle Swarms, Ant Colony, Bacterial Foraging, Artificial Bees, Fireflies Algorithm, Harmony Search, Artificial Life, Digital Organisms, Estimation of Distribution Algorithms, Stochastic Diffusion Search, Quantum Computing, Nano Computing, Membrane Computing, Human-centric Computing, Hybridization of Algorithms, Memetic Computing, Autonomic Computing, Self-organizing systems, Combinatorial, Discrete, Binary, Constrained, Multi-objective, Multi-modal, Dynamic, and Large-scale Optimization.