With “crisis” comes opportunity: Using invariance testing to understand and improve measurement models (A response to Al-Hoorie et al., 2024)

Abstract

Al-Hoorie et al. (2024) have described L2 Motivational Self System research as having a “validation crisis.” In this response, I offer a brief history of recent motivation research to contextualize how we reached this point. I then discuss measurement invariance testing, the purpose of which is to ensure that the proposed measurement model holds across groups or time. Crucially, invariance testing is a necessary precursor to subsequent analyses because if invariance is not upheld, it is impossible to know whether differences reflect true underlying differences in the latent construct or are simply the byproduct of a changing measurement model. I argue that by combining measurement invariance testing with open science practices, we can create and validate better measurement models while cultivating a better understanding of their scope of application.