Explaining Performance Gaps with Problem-Solving Process Data via Latent Class Mediation Analysis.

Abstract

Process data, in particular, log data collected from a computerized test, documents the sequence of actions performed by an examinee in pursuit of solving a problem, affording an opportunity to understand test-taking behavioral patterns that account for demographic group differences in key outcomes of interest, for instance, final score on a cognitive item. Addressing this aim, this article proposes a latent class mediation analysis procedure. Using continuous process features extracted from action sequence data as indicators, latent classes underlying the test-taking behavior are identified in a latent class mediation model, where an examinee's nominal latent class membership enters as the mediator between the observed grouping and outcome variables. A headlong search algorithm for selecting the subset of process features that maximizes the total indirect effect of the latent class mediator is implemented. The proposed procedure is validated with a series of simulations. An application to a large-scale assessment highlights how the proposed method can be used to explain performance gaps between students with learning disability and their typically developing peers on the National Assessment of Educational Progress (NAEP) math assessment.