Question-order effects on judgements under uncertainty

Abstract

Data on judgements under uncertainty (e.g., expectations and risk preferences) have been increasingly used to predict and understand people's decision making. However, little is known about how sensitive the elicitation of such measures is to questionnaire design. Using a survey experiment conducted in Chile, we assess order effects on the elicitation of expectations and risk preference measures. To do so, we randomly alter the position of questionnaire modules. Our findings suggest that expectations about future work, school, and wellbeing-related outcomes, as well as risk preferences, are sensitive to order effects. When people are asked about their past experiences immediately before the elicitation of such measures, more pessimistic expectations about future outcomes are reported, as well as higher levels of risk aversion. We also observed that positioning non-cognitive skill questions immediately before the elicitation of expectations and risk preferences generates data with lower variance suggesting more reliable measurements. Indeed, using these measures in prediction models of schooling and labour market outcomes, we find higher predictive power of future schooling and labour market outcomes when our measures are preceded by non-cognitive skills questions.