Robust importance sampling with adaptive winsorization

Abstract

Importance sampling is a widely used technique to estimate properties of a distribution. The resulting estimator is unbiased but may have huge, potentially infinite, variance. This paper proposes trading-off some bias for variance by adaptively winsorizing the importance sampling estimator. The novel procedure is based on the Balancing Principle (or Lepskii’s Method). As a consequence, it offers a principled way to perform winsorization with finitesample guarantees in the form of an oracle inequality. In various examples, the proposed estimator is shown to have smaller mean squared error and mean absolute deviation than leading alternatives such as the traditional importance sampling estimator or winsorizing it via cross-validation.