{"title":"Adaptive Q-Learning Based Model-Free $H_{\\infty }$ Control of Continuous-Time Nonlinear Systems: Theory and Application","authors":"Jun Zhao;Yongfeng Lv;Zhangu Wang;Ziliang Zhao","doi":"10.1109/TETCI.2024.3449870","DOIUrl":null,"url":null,"abstract":"Although model based <inline-formula><tex-math>$H_{\\infty }$</tex-math></inline-formula> control scheme for nonlinear continuous-time (CT) systems with unknown system dynamics has been extensively studied, model-free <inline-formula><tex-math>$H_{\\infty }$</tex-math></inline-formula> control of <italic>nonlinear CT systems</i> via Q-learning is still a challenging problem. This paper develops a novel Q-learning based model-free <inline-formula><tex-math>$H_{\\infty }$</tex-math></inline-formula> control scheme for nonlinear CT systems, where the adaptive critic and actor continuously and simultaneously update each other, eliminating the need for iterative steps. As a result, a hybrid structure is avoided and there is no longer a requirement for an initial stabilizing control policy. To obtain the <inline-formula><tex-math>$H_{\\infty }$</tex-math></inline-formula> control of the CT nonlinear system, the Q-learning strategy is introduced to online resolve the <inline-formula><tex-math>$H_{\\infty }$</tex-math></inline-formula> control problem in a non-iterative approach, where the system dynamics are not required. In addition, a new learning law is further developed by utilizing a sliding mode scheme to online update the critic neural network (NN) weights. Due to the strong convergence of critic NN weights, the actor NN used in most <inline-formula><tex-math>$H_{\\infty }$</tex-math></inline-formula> control algorithms is removed. Finally, numerical simulation and experimental results of an adaptive cruise control (ACC) system based on a real vehicle effectively demonstrate the feasibility of the presented control method and learning algorithm.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"9 2","pages":"1143-1152"},"PeriodicalIF":5.3000,"publicationDate":"2024-09-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Emerging Topics in Computational Intelligence","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10663220/","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0
Abstract
Although model based $H_{\infty }$ control scheme for nonlinear continuous-time (CT) systems with unknown system dynamics has been extensively studied, model-free $H_{\infty }$ control of nonlinear CT systems via Q-learning is still a challenging problem. This paper develops a novel Q-learning based model-free $H_{\infty }$ control scheme for nonlinear CT systems, where the adaptive critic and actor continuously and simultaneously update each other, eliminating the need for iterative steps. As a result, a hybrid structure is avoided and there is no longer a requirement for an initial stabilizing control policy. To obtain the $H_{\infty }$ control of the CT nonlinear system, the Q-learning strategy is introduced to online resolve the $H_{\infty }$ control problem in a non-iterative approach, where the system dynamics are not required. In addition, a new learning law is further developed by utilizing a sliding mode scheme to online update the critic neural network (NN) weights. Due to the strong convergence of critic NN weights, the actor NN used in most $H_{\infty }$ control algorithms is removed. Finally, numerical simulation and experimental results of an adaptive cruise control (ACC) system based on a real vehicle effectively demonstrate the feasibility of the presented control method and learning algorithm.
期刊介绍:
The IEEE Transactions on Emerging Topics in Computational Intelligence (TETCI) publishes original articles on emerging aspects of computational intelligence, including theory, applications, and surveys.
TETCI is an electronics only publication. TETCI publishes six issues per year.
Authors are encouraged to submit manuscripts in any emerging topic in computational intelligence, especially nature-inspired computing topics not covered by other IEEE Computational Intelligence Society journals. A few such illustrative examples are glial cell networks, computational neuroscience, Brain Computer Interface, ambient intelligence, non-fuzzy computing with words, artificial life, cultural learning, artificial endocrine networks, social reasoning, artificial hormone networks, computational intelligence for the IoT and Smart-X technologies.