Investigating applicability of ratings of indicators of the CLASS Pre-K instrument

IF 1.5 Q2 EDUCATION & EDUCATIONAL RESEARCH

International Journal of Research & Method in Education Pub Date : 2022-09-29 DOI:10.1080/1743727X.2022.2128741

Eija Pakarinen, L. Malmberg, A. Poikkeus, Martti Siekkinen, Marja‐Kristiina Lerkkanen

{"title":"Investigating applicability of ratings of indicators of the CLASS Pre-K instrument","authors":"Eija Pakarinen, L. Malmberg, A. Poikkeus, Martti Siekkinen, Marja‐Kristiina Lerkkanen","doi":"10.1080/1743727X.2022.2128741","DOIUrl":null,"url":null,"abstract":"ABSTRACT When classroom observations are increasingly used for accountability and evaluation purposes, a deeper understanding of the psychometric properties of such measurement tools is needed. The present study took a unique approach to examine the psychometric properties of a commonly used classroom observation measure by testing the reliability of indicators for higher-order constructs (i.e. dimensions). We investigated the reliability of indicator ratings of the Classroom Assessment Scoring System (CLASS) Pre-K instrument in Finnish kindergarten and first grade classrooms. Twenty-one observer pairs rated 838 segments identified from the 413 lessons of 48 teachers. Variance components models were specified to investigate variance proportions of each indicator and dimension. The results showed that most observer disagreement was found for the instructional support domain. Observers disagreed relatively more depending on the teacher they observed. There is a clear need for additional understanding on how observers process information on the complex elements of classroom interaction in order to improve training programmes and the reliability and accuracy of the assessment procedure.","PeriodicalId":51655,"journal":{"name":"International Journal of Research & Method in Education","volume":null,"pages":null},"PeriodicalIF":1.5000,"publicationDate":"2022-09-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Journal of Research & Method in Education","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1080/1743727X.2022.2128741","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}

引用次数: 0

Abstract

ABSTRACT When classroom observations are increasingly used for accountability and evaluation purposes, a deeper understanding of the psychometric properties of such measurement tools is needed. The present study took a unique approach to examine the psychometric properties of a commonly used classroom observation measure by testing the reliability of indicators for higher-order constructs (i.e. dimensions). We investigated the reliability of indicator ratings of the Classroom Assessment Scoring System (CLASS) Pre-K instrument in Finnish kindergarten and first grade classrooms. Twenty-one observer pairs rated 838 segments identified from the 413 lessons of 48 teachers. Variance components models were specified to investigate variance proportions of each indicator and dimension. The results showed that most observer disagreement was found for the instructional support domain. Observers disagreed relatively more depending on the teacher they observed. There is a clear need for additional understanding on how observers process information on the complex elements of classroom interaction in order to improve training programmes and the reliability and accuracy of the assessment procedure.

查看原文本刊更多论文

调查CLASS Pre-K仪器指标评级的适用性

摘要当课堂观察越来越多地用于问责和评估目的时，需要更深入地了解这些测量工具的心理测量特性。本研究采用了一种独特的方法，通过测试高阶结构（即维度）指标的可靠性，来检验常用课堂观察测量的心理测量特性。我们调查了芬兰幼儿园和一年级教室课堂评估评分系统（CLASS）学前教育工具指标评分的可靠性。21对观察者对48名教师的413节课中确定的838节进行了评分。指定方差分量模型来研究每个指标和维度的方差比例。结果表明，大多数观察者对教学支持领域存在分歧。观察者的分歧相对较大，这取决于他们观察到的老师。显然需要进一步了解观察员如何处理课堂互动复杂要素的信息，以改进培训方案以及评估程序的可靠性和准确性。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

International Journal of Research & Method in Education EDUCATION & EDUCATIONAL RESEARCH-

CiteScore

4.70

自引率

5.00%

发文量

期刊介绍： The International Journal of Research & Method in Education is an interdisciplinary, peer-reviewed journal that draws contributions from a wide community of international researchers. Contributions are expected to develop and further international discourse in educational research with a particular focus on method and methodological issues. The journal welcomes papers engaging with methods from within a qualitative or quantitative framework, or from frameworks which cut across and or challenge this duality. Papers should not solely focus on the practice of education; there must be a contribution to methodology. International Journal of Research & Method in Education is committed to publishing scholarly research that discusses conceptual, theoretical and methodological issues, provides evidence, support for or informed critique of unusual or new methodologies within educational research and provides innovative, new perspectives and examinations of key research findings. The journal’s enthusiasm to foster debate is also recognised in a keenness to include engaged, thought-provoking response papers to previously published articles. The journal is also interested in papers that discuss issues in the teaching of research methods for educational researchers. Contributors to International Journal of Research & Method in Education should take care to communicate their findings or arguments in a succinct, accessible manner to an international readership of researchers, policy-makers and practitioners from a range of disciplines including but not limited to philosophy, sociology, economics, psychology, and history of education. The Co-Editors welcome suggested topics for future Special Issues. Initial ideas should be discussed by email with the Co-Editors before a formal proposal is submitted for consideration.