The Cesàro Value Iteration

Abstract

In this letter, we consider undiscouted infinite-horizon optimal control for deterministic systems with an uncountable state and input space. We specifically address the case when the classic value iteration does not converge. For such systems, we use the Cesàro mean to define the infinite-horizon optimal control problem and the corresponding infinite-horizon value function. Moreover, for this value function, we introduce the Cesàro value iteration and prove its convergence for the special case of systems with periodic optimal operating behavior. For this instance, we also show that the Cesàro value function recovers the undiscounted infinite-horizon optimal cost, if the latter is well-defined.