Evaluating Different Scoring Methods for the Speeded Cloze-elide Test: The Application of the {R}asch Partial Credit Model

IF 1.3

The quantitative methods for psychology Pub Date : 2022-10-01 DOI:10.20982/tqmp.18.3.p241

Farshad Effatpanah, Purya Baghaei

{"title":"Evaluating Different Scoring Methods for the Speeded Cloze-elide Test: The Application of the {R}asch Partial Credit Model","authors":"Farshad Effatpanah, Purya Baghaei","doi":"10.20982/tqmp.18.3.p241","DOIUrl":null,"url":null,"abstract":"Cloze-elide tests are overall measures of both ﬁrst (L1) and second language (L2) reading comprehension and communicative skills. Research has shown that a time constraint is an effective method to understand individual differences and increase the reliability and validity of tests. The purpose of this study is to investigate the psychometric quality of a speeded cloze-elide test using a ploytomous Rasch model, called partial credit model (PCM), by inspecting the ﬁt of four different scoring techniques. To this end, responses of 150 English as a foreign language (EFL) students to a speeded cloze-elide test was analyzed. The comparison of different scoring techniques revealed that scoring based on wrong scores can better explain variability in the data. The results of PCM indicated that the assumptions of unidimensionality holds for the speeded cloze-elide test. However, the results of partial credit analysis of data structure revealed that a number of categories do not increase with category values. Finally, suggestions for further research, to better take advantage of the ﬂexibilities of item response theory and Rasch models for explaining count data, will be presented.","PeriodicalId":93055,"journal":{"name":"The quantitative methods for psychology","volume":" ","pages":""},"PeriodicalIF":1.3000,"publicationDate":"2022-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"The quantitative methods for psychology","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.20982/tqmp.18.3.p241","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 0

Abstract

Cloze-elide tests are overall measures of both ﬁrst (L1) and second language (L2) reading comprehension and communicative skills. Research has shown that a time constraint is an effective method to understand individual differences and increase the reliability and validity of tests. The purpose of this study is to investigate the psychometric quality of a speeded cloze-elide test using a ploytomous Rasch model, called partial credit model (PCM), by inspecting the ﬁt of four different scoring techniques. To this end, responses of 150 English as a foreign language (EFL) students to a speeded cloze-elide test was analyzed. The comparison of different scoring techniques revealed that scoring based on wrong scores can better explain variability in the data. The results of PCM indicated that the assumptions of unidimensionality holds for the speeded cloze-elide test. However, the results of partial credit analysis of data structure revealed that a number of categories do not increase with category values. Finally, suggestions for further research, to better take advantage of the ﬂexibilities of item response theory and Rasch models for explaining count data, will be presented.

查看原文本刊更多论文

快速完形填空不同评分方法的评价——应用{R}asch部分信贷模型

完形填空测试是对第一语言（L1）和第二语言（L2）阅读理解和沟通技能的全面衡量。研究表明，时间约束是理解个体差异、提高测试信度和有效性的有效方法。本研究的目的是通过检查四种不同评分技术的效果，使用一种称为部分信用模型（PCM）的复杂Rasch模型来研究快速完形填空的心理测量质量。为此，我们分析了150名英语学习者对一项快速完形填空的反应。不同评分技术的比较表明，基于错误分数的评分可以更好地解释数据的可变性。PCM的结果表明，一维假设适用于快速完形填空。然而，数据结构的部分信用分析结果显示，许多类别并没有随着类别值的增加而增加。最后，将提出进一步研究的建议，以更好地利用项目反应理论和Rasch模型的灵活性来解释计数数据。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

The quantitative methods for psychology

自引率

0.00%

发文量