Efficient Generalized Ridge Regression

Abstract

Abstract The original ridge estimator of the unknown p×1 vector of β-coefficients in a linear model used a single scalar, k, to determine a point on a shrinkage path of finite length that extends from the Ordinary Least Squares estimator, ^β0, to the shrinkage terminus (usually ^β ≡ 0). Generalized ridge estimators use two or more parameters to determine not only the shape of their shrinkage path but also a specific point on that path. The efficient generalized ridge regression path proposed here is a continuous two-piece linear function that (1) starts at ^β0, the Best Linear Unbiased Estimator, (2) has its interior knot at the ^ β-estimator with Maximum Likelihood of achieving overall minimum MSE risk under normal distribution-theory, and (3) ends at the shrinkage terminus. This new path is efficient in the senses that it is the shortest path and, at least when p > 2, essentially the only known shrinkage path that always contains the ^ β-vector that is most likely to be optimally biased. Functions in R-packages freely distributed via CRAN perform the calculations and produce the graphics used here to illustrate shrinkage concepts by interpreting ridge Trace diagnostic plots. These new concepts and visual tools provide invaluable data-analytic insights and improved self-confidence to applied researchers and data scientists fitting linear models when p, the number of non-constant X-predictor variables in the model, is strictly less than the number of observations available.