{"title":"基于强化学习的离散时间系统数据驱动干扰补偿控制","authors":"Lanyue Li, Jinna Li, Jiangtao Cao","doi":"10.1002/acs.3793","DOIUrl":null,"url":null,"abstract":"SummaryIn this article, a self‐learning disturbance compensation control method is developed, which enables the unknown discrete‐time (DT) systems to achieve performance optimization in the presence of disturbances. Different from traditional model‐based and data‐driven state feedback control methods, the developed off‐policy Q‐learning algorithm updates the state feedback controller parameters and the compensator parameters by actively interacting with the unknown environment, thus the approximately optimal tracking can be realized using only data. First, an optimal tracking problem for a linear DT system with disturbance is formulated. Then, the design for controller is achieved by solving a zero‐sum game problem, leading to an off‐policy disturbance compensation Q‐learning algorithm with only a critic structure, which uses data to update disturbance compensation controller gains, without the knowledge of system dynamics. Finally, the effectiveness of the proposed method is verified by simulations.","PeriodicalId":50347,"journal":{"name":"International Journal of Adaptive Control and Signal Processing","volume":"159 1","pages":""},"PeriodicalIF":3.9000,"publicationDate":"2024-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Data‐driven disturbance compensation control for discrete‐time systems based on reinforcement learning\",\"authors\":\"Lanyue Li, Jinna Li, Jiangtao Cao\",\"doi\":\"10.1002/acs.3793\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"SummaryIn this article, a self‐learning disturbance compensation control method is developed, which enables the unknown discrete‐time (DT) systems to achieve performance optimization in the presence of disturbances. Different from traditional model‐based and data‐driven state feedback control methods, the developed off‐policy Q‐learning algorithm updates the state feedback controller parameters and the compensator parameters by actively interacting with the unknown environment, thus the approximately optimal tracking can be realized using only data. First, an optimal tracking problem for a linear DT system with disturbance is formulated. Then, the design for controller is achieved by solving a zero‐sum game problem, leading to an off‐policy disturbance compensation Q‐learning algorithm with only a critic structure, which uses data to update disturbance compensation controller gains, without the knowledge of system dynamics. Finally, the effectiveness of the proposed method is verified by simulations.\",\"PeriodicalId\":50347,\"journal\":{\"name\":\"International Journal of Adaptive Control and Signal Processing\",\"volume\":\"159 1\",\"pages\":\"\"},\"PeriodicalIF\":3.9000,\"publicationDate\":\"2024-03-22\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"International Journal of Adaptive Control and Signal Processing\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://doi.org/10.1002/acs.3793\",\"RegionNum\":4,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"AUTOMATION & CONTROL SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Journal of Adaptive Control and Signal Processing","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1002/acs.3793","RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"AUTOMATION & CONTROL SYSTEMS","Score":null,"Total":0}
Data‐driven disturbance compensation control for discrete‐time systems based on reinforcement learning
SummaryIn this article, a self‐learning disturbance compensation control method is developed, which enables the unknown discrete‐time (DT) systems to achieve performance optimization in the presence of disturbances. Different from traditional model‐based and data‐driven state feedback control methods, the developed off‐policy Q‐learning algorithm updates the state feedback controller parameters and the compensator parameters by actively interacting with the unknown environment, thus the approximately optimal tracking can be realized using only data. First, an optimal tracking problem for a linear DT system with disturbance is formulated. Then, the design for controller is achieved by solving a zero‐sum game problem, leading to an off‐policy disturbance compensation Q‐learning algorithm with only a critic structure, which uses data to update disturbance compensation controller gains, without the knowledge of system dynamics. Finally, the effectiveness of the proposed method is verified by simulations.
期刊介绍:
The International Journal of Adaptive Control and Signal Processing is concerned with the design, synthesis and application of estimators or controllers where adaptive features are needed to cope with uncertainties.Papers on signal processing should also have some relevance to adaptive systems. The journal focus is on model based control design approaches rather than heuristic or rule based control design methods. All papers will be expected to include significant novel material.
Both the theory and application of adaptive systems and system identification are areas of interest. Papers on applications can include problems in the implementation of algorithms for real time signal processing and control. The stability, convergence, robustness and numerical aspects of adaptive algorithms are also suitable topics. The related subjects of controller tuning, filtering, networks and switching theory are also of interest. Principal areas to be addressed include:
Auto-Tuning, Self-Tuning and Model Reference Adaptive Controllers
Nonlinear, Robust and Intelligent Adaptive Controllers
Linear and Nonlinear Multivariable System Identification and Estimation
Identification of Linear Parameter Varying, Distributed and Hybrid Systems
Multiple Model Adaptive Control
Adaptive Signal processing Theory and Algorithms
Adaptation in Multi-Agent Systems
Condition Monitoring Systems
Fault Detection and Isolation Methods
Fault Detection and Isolation Methods
Fault-Tolerant Control (system supervision and diagnosis)
Learning Systems and Adaptive Modelling
Real Time Algorithms for Adaptive Signal Processing and Control
Adaptive Signal Processing and Control Applications
Adaptive Cloud Architectures and Networking
Adaptive Mechanisms for Internet of Things
Adaptive Sliding Mode Control.