Successive over relaxation for model-free LQR control of discrete-time Markov jump systems

IF 4.8 2区计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS

Automatica Pub Date : 2024-10-25 DOI:10.1016/j.automatica.2024.111919

Wenwu Fan, Junlin Xiong

引用次数: 0

Abstract

This paper aims to solve the model-free linear quadratic regulator problem for discrete-time Markov jump linear systems without requiring an initial stabilizing control policy. We propose both model-based and model-free successive over relaxation algorithms to learn the optimal control policy of discrete-time Markov jump linear systems. The model-free value iteration algorithm is a special case of our model-free algorithm when the relaxation factor equals one. A sufficient condition on the relaxation factor is provided to guarantee the convergence of our algorithms. Moreover, it is proved that our model-free algorithm can obtain an approximate optimal solution when the transition probability matrix is unknown. Finally, a numerical example is used to illustrate our results.

查看原文本刊更多论文

离散时马尔可夫跳跃系统无模型 LQR 控制的连续过度松弛

本文旨在解决离散-时间马尔可夫跳跃线性系统的无模型线性二次调节器问题，而无需初始稳定控制策略。我们提出了基于模型和无模型的迭代松弛算法来学习离散-时间马尔可夫跳跃线性系统的最优控制策略。当松弛因子等于 1 时，无模型值迭代算法是我们无模型算法的特例。我们提供了松弛因子的充分条件，以保证算法的收敛性。此外，还证明了当过渡概率矩阵未知时，我们的无模型算法可以获得近似最优解。最后，我们用一个数值示例来说明我们的结果。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Automatica 工程技术-工程：电子与电气

CiteScore

10.70

自引率

7.80%

发文量

617

审稿时长

5 months

期刊介绍： Automatica is a leading archival publication in the field of systems and control. The field encompasses today a broad set of areas and topics, and is thriving not only within itself but also in terms of its impact on other fields, such as communications, computers, biology, energy and economics. Since its inception in 1963, Automatica has kept abreast with the evolution of the field over the years, and has emerged as a leading publication driving the trends in the field. After being founded in 1963, Automatica became a journal of the International Federation of Automatic Control (IFAC) in 1969. It features a characteristic blend of theoretical and applied papers of archival, lasting value, reporting cutting edge research results by authors across the globe. It features articles in distinct categories, including regular, brief and survey papers, technical communiqués, correspondence items, as well as reviews on published books of interest to the readership. It occasionally publishes special issues on emerging new topics or established mature topics of interest to a broad audience. Automatica solicits original high-quality contributions in all the categories listed above, and in all areas of systems and control interpreted in a broad sense and evolving constantly. They may be submitted directly to a subject editor or to the Editor-in-Chief if not sure about the subject area. Editorial procedures in place assure careful, fair, and prompt handling of all submitted articles. Accepted papers appear in the journal in the shortest time feasible given production time constraints.