Spatial Shrinkage Prior: A Probabilistic Approach to Model for Categorical Variables with Many Levels

Abstract

One of the most commonly used methods to prevent overfitting and select relevant variables in regression models with many predictors is the penalized regression technique. Under such approaches, variable selection is performed in a non-probabilistic way, using some optimization criterion. A Bayesian approach to penalized regression has been proposed by assuming a prior distribution for the regression coefficients that plays a similar role as the penalty term in classical statistics: to shrink non-significant coefficients toward zero and assign a significant probability mass to non-negligible coefficients. These prior distributions, called shrinkage priors, usually assume independence among the covariates, which may not be an appropriate assumption in many cases. We propose two shrinkage priors to model the uncertainty about coefficients that are spatially correlated. The proposed priors are considered as an alternative approach to model the uncertainty about the coefficients of categorical variables with many levels. To illustrate their use, we consider the linear regression model. We evaluate the proposed method through several simulation studies.