Empirical Bayes Model Averaging with Influential Observations: Tuning Zellner’s g Prior for Predictive Robustness

Abstract

The behavior of Bayesian model averaging (BMA) for the normal linear regression model in the presence of influential observations that contribute to model misfit is investigated. Remedies to attenuate the potential negative impacts of such observations on inference and prediction are proposed. The methodology is motivated by the view that well-behaved residuals and good predictive performance often go hand-in-hand. Focus is placed on regression models that use variants on Zellner's $g$ prior. Studying the impact of various forms of model misfit on BMA predictions in simple situations points to prescriptive guidelines for “tuning” Zellner's $g$ prior to obtain optimal predictions. The tuning of the prior distribution is obtained by considering theoretical properties that should be enjoyed by the optimal fits of the various models in the BMA ensemble. The methodology can be thought of as an “empirical Bayes” approach to modeling, as the data help to inform the specification of the prior in an attempt to attenuate the negative impact of influential cases.