The influence of procedural characteristics on within-case effect sizes for academic outcomes

Abstract

Single-case experimental designs (SCEDs) have been used with increasing frequency to identify evidence-based interventions in education. The purpose of this study was to explore how several procedural characteristics, including within-phase variability (i.e., measurement error), number of baseline observations, and number of intervention observations influenced the magnitude of four SCED effect sizes, including (a) non-overlap of all pairs (NAP), (b) baseline corrected tau (BC-Tau), (c) mean-phase difference (MPD), and (d) generalized least squares (GLS) when applied to hypothetical academic intervention SCED data. Higher levels of measurement error decreased the average magnitude of effect sizes, particularly NAP and BC-Tau. However, the number of intervention observations had minimal impact on the average magnitude of NAP and BC-Tau. Increasing the number of intervention observations dramatically increased the magnitude of GLS and MPD. Increasing the number of baseline observations also tended to increase the average magnitude of MPD. The ratio of baseline to intervention observations had a statistically but not practically significant influence on the average magnitude of NAP, BC-Tau, and GLS. Careful consideration is required when determining the length of time academic SCEDs are conducted and what effect sizes are used to summarize treatment outcomes. This article also highlights the value of using meaningful simulation conditions to understand the performance of SCED effect sizes.