Solving optimal predictor-feedback control using approximate dynamic programming

IF 4.8 2区计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS

Automatica Pub Date : 2024-08-31 DOI:10.1016/j.automatica.2024.111848

Hongxia Wang , Fuyu Zhao , Zhaorong Zhang , Juanjuan Xu , Xun Li

{"title":"Solving optimal predictor-feedback control using approximate dynamic programming","authors":"Hongxia Wang , Fuyu Zhao , Zhaorong Zhang , Juanjuan Xu , Xun Li","doi":"10.1016/j.automatica.2024.111848","DOIUrl":null,"url":null,"abstract":"<div><p>This paper is concerned with approximately solving the optimal predictor-feedback control problem of multiplicative-noise systems with input delay in infinite horizon. The optimal predictor-feedback control, provided by the analytical method, is determined by Riccati–ZXL equations and is hard to obtain in the case of unknown system dynamics. We aim to propose a policy iteration (PI) algorithm for solving the optimal solution by approximate dynamic programming. For convergence analysis of the algorithm, we first develop a necessary and sufficient stabilizing condition, in the form of several new Lyapunov-type equations, which parameterizes all predictor-feedback controllers and can be seen as an important addition to Lyapunov stability theory. We then propose an iterative scheme for the Riccati–ZXL equations computations, along with convergence analysis, based on the condition. Inspired by this scheme, a data-driven online PI algorithm, convergence implied in that of the iterative scheme, is proposed for the optimal predictor-feedback control problem without full system dynamics. Finally, a numerical example is used to evaluate the proposed PI algorithm.</p></div>","PeriodicalId":55413,"journal":{"name":"Automatica","volume":"170 ","pages":"Article 111848"},"PeriodicalIF":4.8000,"publicationDate":"2024-08-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Automatica","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S000510982400342X","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"AUTOMATION & CONTROL SYSTEMS","Score":null,"Total":0}

引用次数: 0

Abstract

This paper is concerned with approximately solving the optimal predictor-feedback control problem of multiplicative-noise systems with input delay in infinite horizon. The optimal predictor-feedback control, provided by the analytical method, is determined by Riccati–ZXL equations and is hard to obtain in the case of unknown system dynamics. We aim to propose a policy iteration (PI) algorithm for solving the optimal solution by approximate dynamic programming. For convergence analysis of the algorithm, we first develop a necessary and sufficient stabilizing condition, in the form of several new Lyapunov-type equations, which parameterizes all predictor-feedback controllers and can be seen as an important addition to Lyapunov stability theory. We then propose an iterative scheme for the Riccati–ZXL equations computations, along with convergence analysis, based on the condition. Inspired by this scheme, a data-driven online PI algorithm, convergence implied in that of the iterative scheme, is proposed for the optimal predictor-feedback control problem without full system dynamics. Finally, a numerical example is used to evaluate the proposed PI algorithm.

查看原文本刊更多论文

利用近似动态程序设计解决最优预测反馈控制问题

本文主要研究在无限视距内近似求解具有输入延迟的乘噪声系统的最优预测-反馈控制问题。解析法提供的最优预测反馈控制由 Riccati-ZXL 方程决定，在系统动态未知的情况下很难获得。我们旨在提出一种策略迭代（PI）算法，通过近似动态编程求解最优解。为了分析该算法的收敛性，我们首先以几个新的 Lyapunov 型方程的形式提出了一个必要且充分的稳定条件，该条件涉及所有预测反馈控制器的参数，可视为对 Lyapunov 稳定性理论的重要补充。然后，我们提出了一种基于该条件的 Riccati-ZXL 方程计算迭代方案以及收敛性分析。受这一方案的启发，我们提出了一种数据驱动的在线 PI 算法，其收敛性隐含于迭代方案的收敛性，适用于无完整系统动态的最优预测器-反馈控制问题。最后，通过一个数值示例来评估所提出的 PI 算法。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Automatica 工程技术-工程：电子与电气

CiteScore

10.70

自引率

7.80%

发文量

617

审稿时长

5 months

期刊介绍： Automatica is a leading archival publication in the field of systems and control. The field encompasses today a broad set of areas and topics, and is thriving not only within itself but also in terms of its impact on other fields, such as communications, computers, biology, energy and economics. Since its inception in 1963, Automatica has kept abreast with the evolution of the field over the years, and has emerged as a leading publication driving the trends in the field. After being founded in 1963, Automatica became a journal of the International Federation of Automatic Control (IFAC) in 1969. It features a characteristic blend of theoretical and applied papers of archival, lasting value, reporting cutting edge research results by authors across the globe. It features articles in distinct categories, including regular, brief and survey papers, technical communiqués, correspondence items, as well as reviews on published books of interest to the readership. It occasionally publishes special issues on emerging new topics or established mature topics of interest to a broad audience. Automatica solicits original high-quality contributions in all the categories listed above, and in all areas of systems and control interpreted in a broad sense and evolving constantly. They may be submitted directly to a subject editor or to the Editor-in-Chief if not sure about the subject area. Editorial procedures in place assure careful, fair, and prompt handling of all submitted articles. Accepted papers appear in the journal in the shortest time feasible given production time constraints.