Reliability of Ratings of an English Language Arts Curriculum With the Curriculum Evaluation Guidelines

Abstract

The current study examined the reliability of The Reading League Curriculum-Evaluation Guidelines (CEGs), which were developed to help school-based teams rate the presence of red flags when considering adopting specific literacy curricula. Coders ( n = 30) independently used the CEGs to evaluate a free online English language arts curriculum. The results indicated strong internal consistency ( a = 0.96) and high interrater reliability ( HM = .91, 95% CI = .89 to .93, p < .01). Overall, the CEGs hold the potential as a psychometrically sound tool for evaluating reading curricula. Limitations and implications for practice and research are discussed.