Rasch Model Extensions for Enhanced Formative Assessments in MOOCs

IF 1.1 4区教育学 Q3 EDUCATION & EDUCATIONAL RESEARCH

Applied Measurement in Education Pub Date : 2020-03-03 DOI:10.1080/08957347.2020.1732382

D. Abbakumov, P. Desmet, W. Van den Noortgate

{"title":"Rasch Model Extensions for Enhanced Formative Assessments in MOOCs","authors":"D. Abbakumov, P. Desmet, W. Van den Noortgate","doi":"10.1080/08957347.2020.1732382","DOIUrl":null,"url":null,"abstract":"ABSTRACT Formative assessments are an important component of massive open online courses (MOOCs), online courses with open access and unlimited student participation. Accurate conclusions on students’ proficiency via formative, however, face several challenges: (a) students are typically allowed to make several attempts; and (b) student performance might be affected by other variables, such as interest. Thus, neglecting the effects of attempts and interest in proficiency evaluation might result in biased conclusions. In this study, we try to solve this limitation and propose two extensions of the common psychometric model, the Rasch model, by including the effects of attempts and interest. We illustrate these extensions using real MOOC data and evaluate them using cross-validation. We found that (a) the effects of attempts and interest on the performance are positive on average but both vary among students; (b) a part of the variance in proficiency parameters is due to variation between students in the effect of interest; and (c) the overall accuracy of prediction of student’s item responses using the extensions is 4.3% higher than using the Rasch model.","PeriodicalId":51609,"journal":{"name":"Applied Measurement in Education","volume":"33 1","pages":"113 - 123"},"PeriodicalIF":1.1000,"publicationDate":"2020-03-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1080/08957347.2020.1732382","citationCount":"4","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Applied Measurement in Education","FirstCategoryId":"95","ListUrlMain":"https://doi.org/10.1080/08957347.2020.1732382","RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}

引用次数: 4

Abstract

ABSTRACT Formative assessments are an important component of massive open online courses (MOOCs), online courses with open access and unlimited student participation. Accurate conclusions on students’ proficiency via formative, however, face several challenges: (a) students are typically allowed to make several attempts; and (b) student performance might be affected by other variables, such as interest. Thus, neglecting the effects of attempts and interest in proficiency evaluation might result in biased conclusions. In this study, we try to solve this limitation and propose two extensions of the common psychometric model, the Rasch model, by including the effects of attempts and interest. We illustrate these extensions using real MOOC data and evaluate them using cross-validation. We found that (a) the effects of attempts and interest on the performance are positive on average but both vary among students; (b) a part of the variance in proficiency parameters is due to variation between students in the effect of interest; and (c) the overall accuracy of prediction of student’s item responses using the extensions is 4.3% higher than using the Rasch model.

查看原文本刊更多论文

Rasch模型扩展在mooc中增强形成性评估

形成性评估是大规模开放在线课程(MOOCs)的重要组成部分，mooc是开放获取和无限制学生参与的在线课程。然而，通过形成性测试对学生的熟练程度做出准确的结论面临着几个挑战:(a)学生通常被允许进行多次尝试;(b)学生的表现可能会受到其他变量的影响，比如兴趣。因此，忽视尝试和兴趣对熟练程度评价的影响可能会导致有偏见的结论。在本研究中，我们试图解决这一限制，并提出了两种扩展常见的心理测量模型，Rasch模型，包括尝试和兴趣的影响。我们使用真实的MOOC数据来说明这些扩展，并使用交叉验证来评估它们。我们发现:(a)尝试和兴趣对成绩的影响总体上是积极的，但在学生之间存在差异;(b)熟练程度参数的部分差异是由于学生之间兴趣影响的差异;(c)使用扩展预测学生项目反应的总体准确性比使用Rasch模型高4.3%。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Applied Measurement in Education Multiple-

CiteScore

2.50

自引率

13.30%

发文量

期刊介绍： Because interaction between the domains of research and application is critical to the evaluation and improvement of new educational measurement practices, Applied Measurement in Education" prime objective is to improve communication between academicians and practitioners. To help bridge the gap between theory and practice, articles in this journal describe original research studies, innovative strategies for solving educational measurement problems, and integrative reviews of current approaches to contemporary measurement issues. Peer Review Policy: All review papers in this journal have undergone editorial screening and peer review.