Huaguang Zhang;Lulu Zhang;Jiayue Sun;Tianbiao Wang
{"title":"Optimal Control for Unknown Nonlinear System With Semi-Markovian Jump Parameters via Adaptive Dynamic Programming","authors":"Huaguang Zhang;Lulu Zhang;Jiayue Sun;Tianbiao Wang","doi":"10.1109/TSMC.2024.3421658","DOIUrl":null,"url":null,"abstract":"This article investigates the optimal control problem for the discrete-time (DT) nonlinear semi-Markovian jump systems (s-MJSs) that possess unknown dynamics. The study uses the semi-Markovian kernel approach to address the problem of mode-switching in these systems. This approach employs the transition probability and the sojourn-time distribution function to jointly determine the transitions between different modes. Then, with a neural network (NN) identifier, the demand for accurate information on the system dynamics is eliminated, and an optimal control method for the nonlinear s-MJSs is utilized to solve the Hamilton-Jacobi–Bellman equation (HJBE) built upon adaptive dynamic programming methodology. Additionally, a detailed analysis of the convergence of a value iteration-based algorithm, which solves the optimal control issue for the DT s-MJSs, is thoroughly discussed. Furthermore, an actor-critic NN is trained to attain an estimated solution to the relevant HJBE. Finally, to validate the designed approach, two simulations are performed to prove its effectiveness.","PeriodicalId":48915,"journal":{"name":"IEEE Transactions on Systems Man Cybernetics-Systems","volume":null,"pages":null},"PeriodicalIF":8.6000,"publicationDate":"2024-07-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Systems Man Cybernetics-Systems","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10607965/","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"AUTOMATION & CONTROL SYSTEMS","Score":null,"Total":0}
引用次数: 0
Abstract
This article investigates the optimal control problem for the discrete-time (DT) nonlinear semi-Markovian jump systems (s-MJSs) that possess unknown dynamics. The study uses the semi-Markovian kernel approach to address the problem of mode-switching in these systems. This approach employs the transition probability and the sojourn-time distribution function to jointly determine the transitions between different modes. Then, with a neural network (NN) identifier, the demand for accurate information on the system dynamics is eliminated, and an optimal control method for the nonlinear s-MJSs is utilized to solve the Hamilton-Jacobi–Bellman equation (HJBE) built upon adaptive dynamic programming methodology. Additionally, a detailed analysis of the convergence of a value iteration-based algorithm, which solves the optimal control issue for the DT s-MJSs, is thoroughly discussed. Furthermore, an actor-critic NN is trained to attain an estimated solution to the relevant HJBE. Finally, to validate the designed approach, two simulations are performed to prove its effectiveness.
期刊介绍:
The IEEE Transactions on Systems, Man, and Cybernetics: Systems encompasses the fields of systems engineering, covering issue formulation, analysis, and modeling throughout the systems engineering lifecycle phases. It addresses decision-making, issue interpretation, systems management, processes, and various methods such as optimization, modeling, and simulation in the development and deployment of large systems.