How to Understand Common Patterns in Big Data: The Case of Human Collective Memory

Abstract

Simple patterns often arise from complex systems. For example, human perception of similarity decays exponentially with perceptual distance. The ranking of word usage versus the frequency at which the words are used has a log-log slope of minus one. Recent advances in big data provide an opportunity to characterize the commonly observed patterns of nature. Those observed regularities set the challenge of understanding the mechanistic processes that generate common patterns. This article illustrates the problem with the recent big data analysis of collective memory. Collective memory follows a simple biexponential pattern of decay over time. An initial rapid decay is followed by a slower, longer lasting decay. Candia et al. successfully fit a two stage model of mechanistic process to that pattern. Although that fit is useful, this article emphasizes the need, in big data analyses, to consider a broad set of alternative causal explanations. In this case, the method of signal frequency analysis yields several simple alternative models that generate exactly the same observed pattern of collective memory decay. This article concludes that the full potential of big data will require better methods for developing alternative, empirically testable causal models.