Parameter Estimation With Out-of-Sample Objective

Abstract

We study parameter estimation from the sample X, when the objective is to maximize the expected value of a criterion function, Q, for a distinct sample, Y. This is the situation that arises when a model is estimated for the purpose of describing other data than those used for estimation. The motivated for much estimation has this form, with forecasting problems being a prime example. A natural estimator is the innate estimator that maximizes Q(X;\theta.) wrt. \theta. While the innate estimator has certain advantages, we show that the asymptotically efficient estimator is defined from a likelihood function in conjunction with Q. The likelihood-based estimator is, however, fragile, as misspecification is harmful in two ways. First, the likelihood-based estimator may be inefficient under misspecification. Second, and more importantly, the likelihood approach requires a parameter transformation that depends on the truth, causing an improper mapping to be used under misspecification. The theoretical results are illustrated with two applications comprising asymmetric loss and multi-step forecasting, respectively.