Scoring assessments in multisite randomized control trials: Examining the sensitivity of treatment effect estimates to measurement choices.

Abstract

While a great deal of thought, planning, and money goes into the design of multisite randomized control trials (RCTs) that are used to evaluate the effectiveness of interventions in fields like education and psychology, relatively little thought is often paid to the measurement choices made in such evaluations. In this study, we conduct a series of simulation studies that consider a wide range of options for producing scores from multiple administration of assessments in the context of multisite RCTs. The scoring models considered range from the simple (sum scores) to highly complex (multilevel two-tier item response theory [IRT] models with latent regression). We find that the true treatment effect is attenuated when sum scores or scores from IRT models that do not account for treatment assignment are used. (PsycInfo Database Record (c) 2023 APA, all rights reserved).