Extremizing and Anti-Extremizing in Bayesian Ensembles of Binary-Event Forecasts

Abstract

Many organizations combine forecasts of probabilities of binary events to support critical business decisions, such as the approval of credit or the recommendation of a drug. To aggregate individual probabilities, we offer a new method based on Bayesian principles that can help identify why and when combined probabilities need to be extremized. Extremizing is typically viewed as shifting the average probability farther from one half; we emphasize that it is more suitable to define extremizing as shifting it farther from the base rate. We introduce the notion of antiextremizing, cases in which it might be beneficial to make average probabilities less extreme. Analytically, we find that our Bayesian ensembles often extremize the average forecast but sometimes antiextremize instead. On several publicly available data sets, we demonstrate that our Bayesian ensemble performs well and antiextremizes anywhere from 18% to 73% of the cases. Antiextremizing is required more often when there is bracketing with respect to the base rate among the probabilities being aggregated than with no bracketing.