Reviewing the Test Reviews: Quality Judgments and Reviewer Agreements in the Mental Measurements Yearbook

IF 1.1 4区教育学 Q3 EDUCATION & EDUCATIONAL RESEARCH

Applied Measurement in Education Pub Date : 2021-02-25 DOI:10.1080/08957347.2021.1890742

T. Hogan, Marissa DeStefano, Caitlin Gilby, Dana C. Kosman, Joshua Peri

引用次数: 0

Abstract

ABSTRACT Buros’ Mental Measurements Yearbook (MMY) has provided professional reviews of commercially published psychological and educational tests for over 80 years. It serves as a kind of conscience for the testing industry. For a random sample of 50 entries in the 19th MMY (a total of 100 separate reviews) this study determined the level of qualitative judgment rendered by reviewers and the consistency of those independent reviewers in rendering judgments. Judgments of quality distributed themselves almost uniformly from very good to very bad across the 100 reviews. Agreement among reviewers for a given test was positive but relatively weak. We explore implications of the results and suggest follow-up investigations.

查看原文本刊更多论文

复习测验复习：心理测量年鉴中的质量判断和复习者协议

Buros的《心理测量年鉴》（MMY）为商业出版的心理和教育测试提供了80多年的专业评论。这是测试行业的一种良知。本研究随机抽取了第19个MMY的50个条目（共100篇独立评论），确定了评审员的定性判断水平以及这些独立评审员在做出判断时的一致性。在100条评论中，对质量的判断几乎是一致的，从非常好到非常差。评审人员对某项测试的一致性是积极的，但相对较弱。我们探讨了研究结果的影响，并建议进行后续调查。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Applied Measurement in Education Multiple-

CiteScore

2.50

自引率

13.30%

发文量

期刊介绍： Because interaction between the domains of research and application is critical to the evaluation and improvement of new educational measurement practices, Applied Measurement in Education" prime objective is to improve communication between academicians and practitioners. To help bridge the gap between theory and practice, articles in this journal describe original research studies, innovative strategies for solving educational measurement problems, and integrative reviews of current approaches to contemporary measurement issues. Peer Review Policy: All review papers in this journal have undergone editorial screening and peer review.