Stability of infinite-horizon optimal control with discounted cost

Abstract

We investigate the stability of general nonlinear discrete-time systems controlled by an optimal sequence of inputs that minimizes an infinite-horizon discounted cost. We first provide conditions under which a global asymptotic stability property is ensured for the corresponding undiscounted problem. We then show that this property is semiglobally and practically preserved in the discounted case, where the adjustable parameter is the discount factor. We then focus on a scenario where the stage cost is bounded and we explain how our framework applies to guarantee stability in this case. Finally, we provide sufficient conditions, including boundedness of the stage cost, under which the value function, which serves as a Lyapunov function for the analysis, is continuous. As already shown in the literature, the continuity of the Lyapunov function is crucial to ensure some nominal robustness for the closed-loop system.