Prior Dependence in L1-regularized Bayesian Regression

The regularization of regression coefficients has become a central component of research in the statistical sciences due to its importance in applied data analysis in many other fields of science. From a Bayesian perspective, regularization is imposed naturally via prior distributions that probabilistically penalize large values of the coefficients. Research into prior distributions with connections to L1-norm penalization (e.g., “Bayesian lasso” and the “Bayesian elastic net”) has generated important insights about the nature of Bayesian penalized regression in practice. Though widely used, many such priors are restricted by the assumption that the regression coefficients are a priori independent. While independence may be reasonable in some data-analytic settings, having the ability to incorporate dependence in these prior distributions would allow for greater modeling flexibility. I describe a general class of “orthant normal” priors for regression coefficients that allows for prior dependence between regression coefficients. An interesting special case is an L1-regularized version of Zellner’s g prior. Though simulation-based posterior inference via Markov chain Monte Carlo methods is made difficult by an intractable function in the posterior density, I discuss computationally efficient methods for estimating this function that allow for full posterior inference about all model parameters.