Mixed-effects regression weights for advice taking and related phenomena of information sampling and utilization

Abstract

Advice taking and related research is dominated by deterministic weighting indices, specifically ratio-of-differences-based formulas for investigating informational influence. Their arithmetic is intuitively simple, but they pose several measurement problems and restrict research to a particular paradigmatic approach. As a solution, we propose to specify how strongly peoples' judgments are influenced by externally provided evidence by fitting corresponding mixed-effects regression models. Our approach explicitly distinguishes between endogenous components, such as updated beliefs, and exogenous components, such as independent initial judgments and advice. Crucially, mixed-effects regression coefficients of various exogenous sources of information also reflect individual weighting but are based on a conceptually consistent representation of the endogenous judgment process. The formal derivation of the proposed weighting measures is accompanied by a detailed elaboration on their most important technical and statistical subtleties. We use this modeling approach to revisit empirical findings from several paradigms investigating algorithm aversion, sequential collaboration, and advice taking. In summary, we replicate and extend the original finding of algorithm appreciation and initially demonstrate a lack of evidence for both systematic order effects in sequential collaboration as well as differential weighting of multiple pieces of advice. In addition to opening new avenues for innovative research, appropriate modeling of information sampling and utilization has the potential to increase the reproducibility and replicability of behavioral science. Furthermore, the proposed method is relevant beyond advice taking, as mixed-effects regression weights can also inform research on related cognitive phenomena such as multidimensional belief updating, anchoring effects, hindsight bias, or attitude change.