Visualizing Uncertainty: On Soyer's and Hogarth's 'The Illusion of Predictability: How Regression Statistics Mislead Experts'

Abstract

This comment was prepared for the International Journal of Forecasting mini-symposium on the Soyer-Hogarth experiment. The experiment evaluates the ability of expert econometricians to make predictions based on commonly provided regression output. Visual displays of quantitative information, including simple plots of data, outperformed predictions based on R-squared, t-statistics, and other common diagnostics. Reliance on graphing - on the visualization of uncertainty - was suggested more than a century ago by Karl Pearson, a founding father of English language statistics. The results of the Soyer and Hogarth experiment, when combined with evidence produced by Ziliak and McCloskey (2008) and others, suggests that graphing and visualization should receive more attention and tests of statistical significance, less.