Numerical methods for construction of value functions in optimal control problems on an infinite horizon

Abstract

The article is devoted to the analysis of optimal control problems with infinite time horizon. These problems arise in economic growth models and in stabilization problems for dynamic systems. The problem peculiarity is a quality functional with an unbounded integrand which is discounted by an exponential index. The problem is reduced to an equivalent optimal control problem with the stationary value function. It is shown that the value function is the generalized minimax solution of the corresponding Hamilton–Jacobi equation. The boundary condition for the stationary value function is replaced by the property of the Hölder continuity and the sublinear growth condition. A backward procedure on infinite time horizon is proposed for construction of the value function. This procedure approximates the value function as the generalized minimax solution of the stationary Hamilton–Jacobi equation. Its convergence is based on the contraction mapping method defined on the family of uniformly bounded and Hölder continuous functions. After the special change of variables the procedure is realized in numerical finite difference schemes on strongly invariant compact sets for optimal control problems and differential games.