Model selection using cross validation Bayesian predictive densities

Abstract

In this paper, a new model selection criterion for linear Gaussian models is proposed. The criterion is based on choosing the model that achieves the highest prediction ability. A natural way to measure the prediction ability of a given model is to use the principle of cross validation (CV) that partitions the data into estimation set and validation set. However, instead of using CV to obtain a point estimate of the prediction error, the predictive density is used to obtain a measure of the marginal likelihood of the validation data set, conditioned on the event that the estimation data set is observed and that the candidate model is true. The performance of the new criterion is compared with AIC and MDL through Monte Carlo simulations. The cross validation Bayesian predictive density selection rule is shown to outperform the well known consistent criterion MDL. as well as having a good small sample performance.