Reflections on Breiman's Two Cultures of Statistical Modeling

Abstract:In his article on Two Cultures of Statistical Modeling, Leo Breiman argued for an algorithmic approach to statistics, as exemplified by his pathbreaking research on large regularized models that fit data and have good predictive properties but without attempting to capture true underlying structure. I think Breiman was right about the benefits of open-ended predictive methods for complex modern problems. I also discuss some points of disagreement, notably Breiman's dismissal of Bayesian methods, which I think reflected a misunderstanding on his part, in that he did not recognized that Bayesian inference can be viewed as regularized prediction and does not rely on an assumption that the fitted model is true. In retrospect, we can learn both from Breiman's deep foresight and from his occasional oversights.