Ecological cognitive assessment has incremental validity for predicting academic performance over and above single occasion cognitive assessments

Abstract

Throughout the history of psychometric testing, cognitive ability has predominantly been construed as an ability that differs between individuals but not within individuals. As a result, the influence of time and context on within-person variation in cognitive performance has not been well explored. This is despite the fact that the criterion outcomes that cognitive assessments are used to predict, such as educational and workplace performance, inherently encompass performance variability over time and context. In this paper, we provided novel evidence that ecological cognitive assessment has incremental validity for predicting academic performance over and above single occasion cognitive assessments. We proposed the use of a cognitive Ecological Momentary Assessment (EMA) approach to generate short-term repeated measures of cognitive performance and recommended a set of parameters that can be used to describe ecological within-person variation in EMA data. We then empirically tested these parameters across two studies in which participants completed a series of cognitive tasks delivered via the EMA as well as a traditional single occasion cognitive assessment that is time and context-invariant. Our findings showed that a range of ecological cognitive performance parameters had incremental utility for predicting first year university performance above and beyond the traditional cognitive assessment. Further, this appeared to occur because ecological cognitive performance parameters describe some aspect of cognitive performance not captured by the traditional time and context-invariant assessment. We suggest that what these parameters capture is information about an individual's typical ecological cognitive performance over the short-term, which is critical information for predicting educational and workplace success.