用课堂观察评估教师效能:校长评鉴效果之Rasch分析

IF 2.1 Q1 EDUCATION & EDUCATIONAL RESEARCH

Educational Assessment Pub Date : 2019-01-09 DOI:10.1080/10627197.2018.1564272

E. Jones, C. Bergin

{"title":"用课堂观察评估教师效能:校长评鉴效果之Rasch分析","authors":"E. Jones, C. Bergin","doi":"10.1080/10627197.2018.1564272","DOIUrl":null,"url":null,"abstract":"ABSTRACT In most U.S. schools, teachers are evaluated using observation of teaching practice (OTP). This study investigates rater effects on OTP ratings among 421 principals in an authentic teacher evaluation system. Many-facet Rasch analysis (MFR) using a block of shared ratings revealed that principals generally (a) differentiated between more and less effective teachers, (b) rated their teachers with leniency (i.e., overused higher rating categories), and (c) differentiated between teaching practices (e.g., Cognitive Engagement vs. Classroom Management) with minimal halo effect. Individual principals varied significantly in degree of leniency, and approximately 12% of principals exhibited severe rater bias. Implications for use of OTP ratings for evaluating teachers’ effectiveness are discussed. Strengths and limitations of MFR to analyze rater effects in OTP are also discussed.","PeriodicalId":46209,"journal":{"name":"Educational Assessment","volume":"24 1","pages":"118 - 91"},"PeriodicalIF":2.1000,"publicationDate":"2019-01-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1080/10627197.2018.1564272","citationCount":"12","resultStr":"{\"title\":\"Evaluating Teacher Effectiveness Using Classroom Observations: A Rasch Analysis of the Rater Effects of Principals\",\"authors\":\"E. Jones, C. Bergin\",\"doi\":\"10.1080/10627197.2018.1564272\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"ABSTRACT In most U.S. schools, teachers are evaluated using observation of teaching practice (OTP). This study investigates rater effects on OTP ratings among 421 principals in an authentic teacher evaluation system. Many-facet Rasch analysis (MFR) using a block of shared ratings revealed that principals generally (a) differentiated between more and less effective teachers, (b) rated their teachers with leniency (i.e., overused higher rating categories), and (c) differentiated between teaching practices (e.g., Cognitive Engagement vs. Classroom Management) with minimal halo effect. Individual principals varied significantly in degree of leniency, and approximately 12% of principals exhibited severe rater bias. Implications for use of OTP ratings for evaluating teachers’ effectiveness are discussed. Strengths and limitations of MFR to analyze rater effects in OTP are also discussed.\",\"PeriodicalId\":46209,\"journal\":{\"name\":\"Educational Assessment\",\"volume\":\"24 1\",\"pages\":\"118 - 91\"},\"PeriodicalIF\":2.1000,\"publicationDate\":\"2019-01-09\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://sci-hub-pdf.com/10.1080/10627197.2018.1564272\",\"citationCount\":\"12\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Educational Assessment\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1080/10627197.2018.1564272\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"EDUCATION & EDUCATIONAL RESEARCH\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Educational Assessment","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1080/10627197.2018.1564272","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}

引用次数: 12

摘要

摘要在大多数美国学校，教师的评估都采用教学实践观察法（OTP）。本研究在一个真实的教师评估系统中调查了421名校长的评分者对OTP评分的影响。使用共享评级块的多方面Rasch分析（MFR）显示，校长通常（a）区分效率较高和较低的教师，（b）对教师的评级较宽（即过度使用较高评级类别），以及（c）区分光环效应最小的教学实践（如认知参与与课堂管理）。个别校长的宽大程度差异很大，约12%的校长表现出严重的评分者偏见。讨论了使用OTP评分评估教师效能的意义。还讨论了MFR分析OTP中评分者效应的优势和局限性。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Evaluating Teacher Effectiveness Using Classroom Observations: A Rasch Analysis of the Rater Effects of Principals

ABSTRACT In most U.S. schools, teachers are evaluated using observation of teaching practice (OTP). This study investigates rater effects on OTP ratings among 421 principals in an authentic teacher evaluation system. Many-facet Rasch analysis (MFR) using a block of shared ratings revealed that principals generally (a) differentiated between more and less effective teachers, (b) rated their teachers with leniency (i.e., overused higher rating categories), and (c) differentiated between teaching practices (e.g., Cognitive Engagement vs. Classroom Management) with minimal halo effect. Individual principals varied significantly in degree of leniency, and approximately 12% of principals exhibited severe rater bias. Implications for use of OTP ratings for evaluating teachers’ effectiveness are discussed. Strengths and limitations of MFR to analyze rater effects in OTP are also discussed.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

Educational Assessment EDUCATION & EDUCATIONAL RESEARCH-

CiteScore

3.20

自引率

6.70%

发文量

期刊介绍： Educational Assessment publishes original research and scholarship on the assessment of individuals, groups, and programs in educational settings. It includes theory, methodological approaches and empirical research in the appraisal of the learning and achievement of students and teachers, young children and adults, and novices and experts. The journal reports on current large-scale testing practices, discusses alternative approaches, presents scholarship on classroom assessment practices and includes assessment topics debated at the national level. It welcomes both conceptual and empirical pieces and encourages articles that provide a strong bridge between theory and/or empirical research and the implications for educational policy and/or practice.