Discrete-Time Approximation of Stochastic Optimal Control with Partial Observation

IF 2.2 2区数学 Q2 AUTOMATION & CONTROL SYSTEMS

SIAM Journal on Control and Optimization Pub Date : 2024-01-23 DOI:10.1137/23m1549018

Yunzhang Li, Xiaolu Tan, Shanjian Tang

{"title":"Discrete-Time Approximation of Stochastic Optimal Control with Partial Observation","authors":"Yunzhang Li, Xiaolu Tan, Shanjian Tang","doi":"10.1137/23m1549018","DOIUrl":null,"url":null,"abstract":"SIAM Journal on Control and Optimization, Volume 62, Issue 1, Page 326-350, February 2024. <br/> Abstract. We consider a class of stochastic optimal control problems with partial observation, and study their approximation by discrete-time control problems. We establish a convergence result by using the weak convergence technique of Kushner and Dupuis [Numerical Methods for Stochastic Control Problems in Continuous Time, Springer, New York], together with the notion of relaxed control rule introduced by El Karoui, Huù Nguyen and Jeanblanc-Picqué [SIAM J. Control Optim., 26 (1988), pp. 1025–1061]. In particular, with a well chosen discrete-time control system, we obtain a first implementable numerical algorithm (with convergence) for the partially observed control problem. Moreover, our discrete-time approximation result would open the door to study convergence of more general numerical approximation methods, such as machine learning based methods. Finally, we illustrate our convergence result by numerical experiments on a partially observed control problem in a linear quadratic setting.","PeriodicalId":49531,"journal":{"name":"SIAM Journal on Control and Optimization","volume":null,"pages":null},"PeriodicalIF":2.2000,"publicationDate":"2024-01-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"SIAM Journal on Control and Optimization","FirstCategoryId":"100","ListUrlMain":"https://doi.org/10.1137/23m1549018","RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"AUTOMATION & CONTROL SYSTEMS","Score":null,"Total":0}

引用次数: 0

Abstract

SIAM Journal on Control and Optimization, Volume 62, Issue 1, Page 326-350, February 2024.
Abstract. We consider a class of stochastic optimal control problems with partial observation, and study their approximation by discrete-time control problems. We establish a convergence result by using the weak convergence technique of Kushner and Dupuis [Numerical Methods for Stochastic Control Problems in Continuous Time, Springer, New York], together with the notion of relaxed control rule introduced by El Karoui, Huù Nguyen and Jeanblanc-Picqué [SIAM J. Control Optim., 26 (1988), pp. 1025–1061]. In particular, with a well chosen discrete-time control system, we obtain a first implementable numerical algorithm (with convergence) for the partially observed control problem. Moreover, our discrete-time approximation result would open the door to study convergence of more general numerical approximation methods, such as machine learning based methods. Finally, we illustrate our convergence result by numerical experiments on a partially observed control problem in a linear quadratic setting.

查看原文本刊更多论文

带部分观测的随机最优控制的离散时间逼近

SIAM 控制与优化期刊》第 62 卷第 1 期第 326-350 页，2024 年 2 月。摘要。我们考虑了一类具有部分观测的随机最优控制问题，并研究了它们的离散时间控制问题近似。我们利用 Kushner 和 Dupuis [Numerical Methods for Stochastic Control Problems in Continuous Time, Springer, New York] 的弱收敛技术，以及 El Karoui、Huù Nguyen 和 Jeanblanc-Picqué [SIAM J. Control Optim.特别是，在离散时间控制系统选择良好的情况下，我们首次获得了部分观测控制问题的可实现数值算法（具有收敛性）。此外，我们的离散时间近似结果为研究更一般的数值近似方法（如基于机器学习的方法）的收敛性打开了大门。最后，我们通过对线性二次方程环境下部分观测控制问题的数值实验来说明我们的收敛结果。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

SIAM Journal on Control and Optimization 数学-应用数学

CiteScore

4.00

自引率

4.50%

发文量

143

审稿时长

12 months

期刊介绍： SIAM Journal on Control and Optimization (SICON) publishes original research articles on the mathematics and applications of control theory and certain parts of optimization theory. Papers considered for publication must be significant at both the mathematical level and the level of applications or potential applications. Papers containing mostly routine mathematics or those with no discernible connection to control and systems theory or optimization will not be considered for publication. From time to time, the journal will also publish authoritative surveys of important subject areas in control theory and optimization whose level of maturity permits a clear and unified exposition. The broad areas mentioned above are intended to encompass a wide range of mathematical techniques and scientific, engineering, economic, and industrial applications. These include stochastic and deterministic methods in control, estimation, and identification of systems; modeling and realization of complex control systems; the numerical analysis and related computational methodology of control processes and allied issues; and the development of mathematical theories and techniques that give new insights into old problems or provide the basis for further progress in control theory and optimization. Within the field of optimization, the journal focuses on the parts that are relevant to dynamic and control systems. Contributions to numerical methodology are also welcome in accordance with these aims, especially as related to large-scale problems and decomposition as well as to fundamental questions of convergence and approximation.