Continuity and robustness to incorrect priors in estimation and control

Abstract

This paper studies continuity properties of single and multi stage estimation and stochastic control problems with respect to initial probability distributions and applications of these results to the study of robustness of control policies applied to systems with incomplete probabilistic models. We establish that continuity and robustness cannot be guaranteed under weak and setwise convergences, but the optimal cost is continuous under the more stringent topology of total variation for stage-wise cost functions that are nonnegative, measurable, and bounded. Under further conditions on either the measurement channels or the source processes, however, weak convergence is sufficient. We also discuss similar properties under the Wasserstein distance. These results are shown to have direct implications, positive or negative, for robust control: If an optimal control policy is applied to a prior model P̃, and if P̃ is close to the true model P, then the application of the incorrect optimal policy to the true model leads to a loss that is continuous in the distance between P̃ and P under total variation, and under some setups, weak convergence distance measures.