Diffusive limit approximation of pure jump optimal ergodic control problems

Abstract

Motivated by the design of fast reinforcement learning algorithms, see (Croissant et al., 2024), we study the diffusive limit of a class of pure jump ergodic stochastic control problems. We show that, whenever the intensity of jumps ${\varepsilon }^{-1}$ is large enough, the approximation error is governed by the Hölder regularity of the Hessian matrix of the solution to the limit ergodic partial differential equation and is, indeed, of order ${\varepsilon }^{\frac{\gamma }{2}}$ for all $\gamma \in (0,1)$. This extends to this context the results of Abeille et al. (2023) obtained for finite horizon problems. Using the limit as an approximation, instead of directly solving the pre-limit problem, allows for a very significant reduction in the numerical resolution cost of the control problem. Additionally, we explain how error correction terms of this approximation can be constructed under appropriate smoothness assumptions. Finally, we quantify the error induced by the use of the Markov control policy constructed from the numerical finite difference scheme associated to the limit diffusive problem, which seems to be new in the literature and of independent interest.