On the Reliable Identification and Effectiveness of Computer-Based, Pop-Up Glossaries in Large-Scale Assessments

IF 1.1 4区教育学 Q3 EDUCATION & EDUCATIONAL RESEARCH

Applied Measurement in Education Pub Date : 2020-07-27 DOI:10.1080/08957347.2020.1789137

D. Cohen, Alesha D. Ballman, F. Rijmen, Jon Cohen

{"title":"On the Reliable Identification and Effectiveness of Computer-Based, Pop-Up Glossaries in Large-Scale Assessments","authors":"D. Cohen, Alesha D. Ballman, F. Rijmen, Jon Cohen","doi":"10.1080/08957347.2020.1789137","DOIUrl":null,"url":null,"abstract":"ABSTRACT Computer-based, pop-up glossaries are perhaps the most promising accommodation aimed at mitigating the influence of linguistic structure and cultural bias on the performance of English Learner (EL) students on statewide assessments. To date, there is no established procedure for identifying the words that require a glossary for EL students that is sufficiently reliable. In the coding procedure, we developed a method to reliably identify words and phrases that require a glossary. The method developed in the coding procedure was used to provide glossaries for the field-test items of statewide English language arts (ELA) and mathematics assessments across grades 3–11 (Current Study). In the Current Study, we assess the effectiveness and influence on construct validity of a pop-up glossary of the words identified in the coding procedure in a large scale, randomized controlled trial. The results demonstrated that generally the pop-up glossary accommodation was effective for both the ELA and mathematics assessments and did not harm the construct being measured.","PeriodicalId":51609,"journal":{"name":"Applied Measurement in Education","volume":"33 1","pages":"378 - 389"},"PeriodicalIF":1.1000,"publicationDate":"2020-07-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1080/08957347.2020.1789137","citationCount":"2","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Applied Measurement in Education","FirstCategoryId":"95","ListUrlMain":"https://doi.org/10.1080/08957347.2020.1789137","RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}

引用次数: 2

Abstract

ABSTRACT Computer-based, pop-up glossaries are perhaps the most promising accommodation aimed at mitigating the influence of linguistic structure and cultural bias on the performance of English Learner (EL) students on statewide assessments. To date, there is no established procedure for identifying the words that require a glossary for EL students that is sufficiently reliable. In the coding procedure, we developed a method to reliably identify words and phrases that require a glossary. The method developed in the coding procedure was used to provide glossaries for the field-test items of statewide English language arts (ELA) and mathematics assessments across grades 3–11 (Current Study). In the Current Study, we assess the effectiveness and influence on construct validity of a pop-up glossary of the words identified in the coding procedure in a large scale, randomized controlled trial. The results demonstrated that generally the pop-up glossary accommodation was effective for both the ELA and mathematics assessments and did not harm the construct being measured.

查看原文本刊更多论文

论大规模评估中基于计算机的弹出式词汇表的可靠识别和有效性

基于计算机的弹出式词汇表可能是最有前途的适应方式，旨在减轻语言结构和文化偏见对英语学习者(EL)学生在全州评估中的表现的影响。到目前为止，还没有确定的程序来识别需要足够可靠的英语学生词汇表的单词。在编码过程中，我们开发了一种可靠地识别需要词汇表的单词和短语的方法。编码过程中开发的方法用于为全州3-11年级英语语言艺术(ELA)和数学评估的现场测试项目提供词汇表(当前研究)。在本研究中，我们通过一项大规模的随机对照试验，评估了在编码过程中识别的单词的弹出式词汇表的有效性及其对结构效度的影响。结果表明，一般而言，弹出式词汇表适应对ELA和数学评估都是有效的，并且不会损害被测量的结构。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Applied Measurement in Education Multiple-

CiteScore

2.50

自引率

13.30%

发文量

期刊介绍： Because interaction between the domains of research and application is critical to the evaluation and improvement of new educational measurement practices, Applied Measurement in Education" prime objective is to improve communication between academicians and practitioners. To help bridge the gap between theory and practice, articles in this journal describe original research studies, innovative strategies for solving educational measurement problems, and integrative reviews of current approaches to contemporary measurement issues. Peer Review Policy: All review papers in this journal have undergone editorial screening and peer review.