Implications of considering Response Process Data for Greater and Lesser Psychometrics

ABSTRACT This paper characterizes the ways in which increased attention to response process data has implications for psychometrics. To do so, this work draws on two organizing frameworks that have heretofore not been associated: evidence-centered design, and the distinction between greater and lesser statistics. Overlaying these frameworks leads to a conceptualization of greater and lesser psychometrics. This provides a conceptual space for articulating the ways in which response process data have implications for lesser psychometrics through measurement models, as well as greater psychometrics through student models, task models, and the social and personal contexts in which assessment takes place. To illustrate the key points and motivate discussions of where future research is needed, I draw from experiences with three assessments that have involved response process data: a task assessing the measurement of geometric area, a performance-based assessment of computer networking, and an educational video game targeting rational number addition.