A Formative Assessment Observation Protocol to Measure Implementation: Evaluating the Scoring Inference

IF 2.1 Q1 EDUCATION & EDUCATIONAL RESEARCH

Educational Assessment Pub Date : 2020-06-04 DOI:10.1080/10627197.2020.1766957

Christine J. Lyon, Leslie Nabors Oláh, Meghan W. Brenneman

{"title":"A Formative Assessment Observation Protocol to Measure Implementation: Evaluating the Scoring Inference","authors":"Christine J. Lyon, Leslie Nabors Oláh, Meghan W. Brenneman","doi":"10.1080/10627197.2020.1766957","DOIUrl":null,"url":null,"abstract":"ABSTRACT There is currently a lack of instrumentation with sufficient technical quality focused on the implementation of classroom formative assessment at a grain-size appropriate for the provision of feedback to teachers and program developers. This paper details the development of a validity argument for the High-Impact Classroom Assessment Practices observation protocol (HI-CAP), and examines how to begin evaluating one inference in the interpretative argument. We present a conceptual framework for the HI-CAP and then articulate the interpretive argument. Finally we present evidence to evaluate one part of this argument, the scoring inference, using independent ratings of lessons from pairs of observers across 65 lessons in ninth-grade ELA and mathematics which suggest modest evidence for appropriateness, consistency, and preliminary evidence supporting the scoring model. We conclude with a discussion of the strengths and limitations of the current protocol and training procedures and implications for developing a validity argument for other similar protocols.","PeriodicalId":46209,"journal":{"name":"Educational Assessment","volume":"25 1","pages":"288 - 313"},"PeriodicalIF":2.1000,"publicationDate":"2020-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1080/10627197.2020.1766957","citationCount":"2","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Educational Assessment","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1080/10627197.2020.1766957","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}

引用次数: 2

Abstract

ABSTRACT There is currently a lack of instrumentation with sufficient technical quality focused on the implementation of classroom formative assessment at a grain-size appropriate for the provision of feedback to teachers and program developers. This paper details the development of a validity argument for the High-Impact Classroom Assessment Practices observation protocol (HI-CAP), and examines how to begin evaluating one inference in the interpretative argument. We present a conceptual framework for the HI-CAP and then articulate the interpretive argument. Finally we present evidence to evaluate one part of this argument, the scoring inference, using independent ratings of lessons from pairs of observers across 65 lessons in ninth-grade ELA and mathematics which suggest modest evidence for appropriateness, consistency, and preliminary evidence supporting the scoring model. We conclude with a discussion of the strengths and limitations of the current protocol and training procedures and implications for developing a validity argument for other similar protocols.

查看原文本刊更多论文

衡量实施的形成性评估观察协议:评估评分推断

目前缺乏具有足够技术质量的工具，专注于以适合向教师和程序开发人员提供反馈的粒度实施课堂形成性评估。本文详细介绍了高影响力课堂评估实践观察协议(HI-CAP)有效性论证的发展，并研究了如何开始评估解释性论证中的一个推论。我们提出了HI-CAP的概念框架，然后阐明了解释性的论点。最后，我们提出证据来评估这一论点的一部分，即得分推理，使用对九年级语文教育和数学的65节课的观察员的课程进行独立评分，这表明适度的证据表明适当性，一致性和初步证据支持得分模型。最后，我们讨论了当前协议和培训程序的优势和局限性，以及为其他类似协议开发有效性论证的意义。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Educational Assessment EDUCATION & EDUCATIONAL RESEARCH-

CiteScore

3.20

自引率

6.70%

发文量

期刊介绍： Educational Assessment publishes original research and scholarship on the assessment of individuals, groups, and programs in educational settings. It includes theory, methodological approaches and empirical research in the appraisal of the learning and achievement of students and teachers, young children and adults, and novices and experts. The journal reports on current large-scale testing practices, discusses alternative approaches, presents scholarship on classroom assessment practices and includes assessment topics debated at the national level. It welcomes both conceptual and empirical pieces and encourages articles that provide a strong bridge between theory and/or empirical research and the implications for educational policy and/or practice.