A bivariate longitudinal cluster model with application to the {C}ognitive {R}eflection {T}est

Abstract

The Cognitive Reflection Test (CRT) is a test designed to assess subjects’ ability to override intuitively appealing but incorrect responses. Psychologists are concerned with whether subjects improve their scores on the test with repeated exposure, in which case, the test’s predictive validity may be threatened. In this paper, we take a novel approach to modelling data recorded on subjects who took the CRT multiple times. We develop bivariate, longitudinal models to describe the responses, CRT score and time taken to complete the CRT. These responses serve as a proxy for the underlying latent variables “numeracy” and “reflectiveness”, respectively—two components of “rationality”. Our models allow for subpopulations of individuals whose responses exhibit similar patterns. We assess the reasonableness of our models via new visualizations of the data. We estimate their parameters by modifying the method of adaptive Gaussian quadrature. We then use our fitted models to address a range of subject-specific questions in a formal way. We find evidence of at least three subpopulations, which we interpret as representing individuals with differing combinations of numeracy and reflectiveness, and determine that, in some subpopulations, test exposure has a greater estimated effect on test scores than previously reported.