On the descriptive value of the reliance on small-samples assumption

Abstract

Experience is the best teacher. Yet, in the context of repeated decisions, experience was found to trigger deviations from maximization in the direction of underweighting of rare events. Evaluations of alternative explanations for this bias led to contradicting conclusions. Studies that focused on the aggregate choice rates, including a series of choice prediction competitions, favored the assumption that this bias reflects reliance on small samples. In contrast, studies that focused on individual decisions suggest that the bias reflects a strong myopic tendency by a significant minority of participants. The current analysis clarifies the apparent inconsistency by reanalyzing a data set that previously led to contradicting conclusions. Our analysis suggests that the apparent inconsistency reflects the differing focus of the cognitive models. Specifically, sequential adjustment models (that assume sensitivity to the payoffs’ weighted averages) tend to find support for the hypothesis that the deviations from maximization are a product of strong positive recency (a form of myopia). Conversely, models assuming random sampling of past experiences tend to find support to the hypothesis that the deviations reflect reliance on small samples. We propose that the debate should be resolved by using the assumptions that provide better predictions. Applying this solution to the data set we analyzed shows that the random sampling assumption outperforms the weighted average assumption both when predicting the aggregate choice rates and when predicting the individual decisions.