Randomized Optimal Stopping Problem in Continuous Time and Reinforcement Learning Algorithm

IF 2.2 2区数学 Q2 AUTOMATION & CONTROL SYSTEMS

SIAM Journal on Control and Optimization Pub Date : 2024-06-03 DOI:10.1137/22m1516725

Yuchao Dong

引用次数: 0

Abstract

SIAM Journal on Control and Optimization, Volume 62, Issue 3, Page 1590-1614, June 2024.
Abstract. In this paper, we study the optimal stopping problem in the so-called exploratory framework, in which the agent takes actions randomly conditioning on the current state and a regularization term is added to the reward functional. Such a transformation reduces the optimal stopping problem to a standard optimal control problem. For the American put option model, we derive the related HJB equation and prove its solvability. Furthermore, we give a convergence rate of policy iteration and compare our solution to the classical American put option problem. Our results indicate a trade-off between the convergence rate and bias in the choice of the temperature constant. Based on the theoretical analysis, a reinforcement learning algorithm is designed and numerical results are demonstrated for several models.

查看原文本刊更多论文

连续时间中的随机最优停止问题和强化学习算法

SIAM 控制与优化期刊》第 62 卷第 3 期第 1590-1614 页，2024 年 6 月。摘要在本文中，我们研究了所谓探索框架下的最优停止问题，即代理根据当前状态随机采取行动，并在奖励函数中加入正则化项。这种转换将最优停止问题简化为一个标准的最优控制问题。对于美式看跌期权模型，我们推导出了相关的 HJB 方程，并证明了其可解性。此外，我们还给出了政策迭代的收敛率，并将我们的解决方案与经典美式看跌期权问题进行了比较。我们的结果表明，在选择温度常数时，收敛速度和偏差之间需要权衡。在理论分析的基础上，我们设计了一种强化学习算法，并对几个模型的数值结果进行了演示。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

SIAM Journal on Control and Optimization 数学-应用数学

CiteScore

4.00

自引率

4.50%

发文量

143

审稿时长

12 months

期刊介绍： SIAM Journal on Control and Optimization (SICON) publishes original research articles on the mathematics and applications of control theory and certain parts of optimization theory. Papers considered for publication must be significant at both the mathematical level and the level of applications or potential applications. Papers containing mostly routine mathematics or those with no discernible connection to control and systems theory or optimization will not be considered for publication. From time to time, the journal will also publish authoritative surveys of important subject areas in control theory and optimization whose level of maturity permits a clear and unified exposition. The broad areas mentioned above are intended to encompass a wide range of mathematical techniques and scientific, engineering, economic, and industrial applications. These include stochastic and deterministic methods in control, estimation, and identification of systems; modeling and realization of complex control systems; the numerical analysis and related computational methodology of control processes and allied issues; and the development of mathematical theories and techniques that give new insights into old problems or provide the basis for further progress in control theory and optimization. Within the field of optimization, the journal focuses on the parts that are relevant to dynamic and control systems. Contributions to numerical methodology are also welcome in accordance with these aims, especially as related to large-scale problems and decomposition as well as to fundamental questions of convergence and approximation.