Equilibrium Data Mining and Data Abundance

Abstract

We model of the search for predictors by speculators (active asset managers) and use it to analyze how the improvement in data processing power and the growth in available data ("data abundance") affect the diversity of trading signals used by speculators, the dispersion of their profits and the similarities of their holdings. Our central message is that data abundance and computing power do not have the same effects. In particular, an improvement in computing power always raises the bar for the quality of predictors that managers consider good enough to exploit while more data lower it when data becomes sufficiently abundant. When this happens, the diversity of speculators' signals and the dispersion of their trading profits increase in equilibrium while their holdings become less correlated.