On generating plausible values for multilevel modelling with large-scale-assessment data

Abstract

Large-scale assessments (LSAs) routinely employ latent regressions to generate plausible values (PVs) for unbiased estimation of the relationship between examinees' background variables and performance. To handle the clustering effect common in LSA data, multilevel modelling is a popular choice. However, most LSAs use single-level conditioning methods, resulting in a mismatch between the imputation model and the multilevel analytic model. While some LSAs have implemented special techniques in single-level latent regressions to support random-intercept modelling, these techniques are not expected to support random-slope models. To address this gap, this study proposed two new single-level methods to support random-slope estimation. The existing and proposed methods were compared to the theoretically unbiased multilevel latent regression method in terms of their ability to support multilevel models. The findings indicate that the two existing single-level methods can support random-intercept-only models. The multilevel latent regression method provided mostly adequate estimates but was limited by computational burden and did not have the best performance across all conditions. One of our proposed single-level methods presented an efficient alternative to multilevel latent regression and was able to recover acceptable estimates for all parameters. We provide recommendations for situations where each method can be applied, with some caveats.