Validity Evidence Supporting Use of Anchoring Vignettes to Measure Teaching Practice

IF 2.1 Q1 EDUCATION & EDUCATIONAL RESEARCH

Educational Assessment Pub Date : 2019-05-27 DOI:10.1080/10627197.2019.1615374

J. Kaufman, J. Engberg, L. Hamilton, Kun Yuan, H. Hill

{"title":"Validity Evidence Supporting Use of Anchoring Vignettes to Measure Teaching Practice","authors":"J. Kaufman, J. Engberg, L. Hamilton, Kun Yuan, H. Hill","doi":"10.1080/10627197.2019.1615374","DOIUrl":null,"url":null,"abstract":"ABSTRACT High-quality measures of instructional practice are essential for research and evaluation of innovative instructional policies and programs. However, existing measures have generally proven inadequate because of cost and validity issues. This paper addresses two potential drawbacks of survey self-report measures: variation in teachers’ interpretation of response scales and their interpretation of survey questions. To address these drawbacks, researchers tested out use of “anchoring vignettes“ in teacher surveys to capture information about teaching practice, and they gathered validity evidence in regard to their use as a tool for adjusting teachers’ survey self-reports about their instructional practices for research purposes, or potentially to inform professional development. Data from 65 teachers in grades 4-9 responding to our survey suggested that vignette adjustments were reliable and valid for some instructional practices more than others. For some instructional practices, researchers found significant and high correlations between teachers’ adjusted survey self-rating, through use of anchoring vignettes, and previous observation ratings of teachers’ instruction, including ratings from several widely-used observation rubrics. These results suggest that anchoring vignettes may provide an efficient, cost-effective method for gathering data on teachers’ instruction.","PeriodicalId":46209,"journal":{"name":"Educational Assessment","volume":null,"pages":null},"PeriodicalIF":2.1000,"publicationDate":"2019-05-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1080/10627197.2019.1615374","citationCount":"5","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Educational Assessment","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1080/10627197.2019.1615374","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}

引用次数: 5

Abstract

ABSTRACT High-quality measures of instructional practice are essential for research and evaluation of innovative instructional policies and programs. However, existing measures have generally proven inadequate because of cost and validity issues. This paper addresses two potential drawbacks of survey self-report measures: variation in teachers’ interpretation of response scales and their interpretation of survey questions. To address these drawbacks, researchers tested out use of “anchoring vignettes“ in teacher surveys to capture information about teaching practice, and they gathered validity evidence in regard to their use as a tool for adjusting teachers’ survey self-reports about their instructional practices for research purposes, or potentially to inform professional development. Data from 65 teachers in grades 4-9 responding to our survey suggested that vignette adjustments were reliable and valid for some instructional practices more than others. For some instructional practices, researchers found significant and high correlations between teachers’ adjusted survey self-rating, through use of anchoring vignettes, and previous observation ratings of teachers’ instruction, including ratings from several widely-used observation rubrics. These results suggest that anchoring vignettes may provide an efficient, cost-effective method for gathering data on teachers’ instruction.

查看原文本刊更多论文

支持使用锚定小片段来衡量教学实践的效度证据

高质量的教学实践措施对于研究和评估创新的教学政策和计划至关重要。然而，由于成本和有效性问题，现有措施通常被证明是不够的。本文讨论了调查自我报告测量的两个潜在缺点：教师对回答量表的解释和对调查问题的解释存在差异。为了解决这些缺点，研究人员测试了“锚定小插曲”的使用“在教师调查中，他们收集了关于教学实践的信息，并收集了有效性证据，证明他们将其作为调整教师调查自我报告的工具，以达到研究目的，或可能为专业发展提供信息。来自65名4-9年级教师的数据表明，小插曲调整是有效的在某些教学实践中比其他教学实践更可靠和有效。对于一些教学实践，研究人员发现，教师通过使用锚定小插曲调整后的调查自我评分与之前对教师教学的观察评分之间存在显著且高度的相关性，包括几种广泛使用的观察准则的评分。这些结果表明，锚定小插曲可以为收集教师教学数据提供一种高效、经济高效的方法。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Educational Assessment EDUCATION & EDUCATIONAL RESEARCH-

CiteScore

3.20

自引率

6.70%

发文量

期刊介绍： Educational Assessment publishes original research and scholarship on the assessment of individuals, groups, and programs in educational settings. It includes theory, methodological approaches and empirical research in the appraisal of the learning and achievement of students and teachers, young children and adults, and novices and experts. The journal reports on current large-scale testing practices, discusses alternative approaches, presents scholarship on classroom assessment practices and includes assessment topics debated at the national level. It welcomes both conceptual and empirical pieces and encourages articles that provide a strong bridge between theory and/or empirical research and the implications for educational policy and/or practice.