How to improve the predictive validity of a composite admission score? A case study from Hungary

IF 4.1 2区 教育学 Q1 EDUCATION & EDUCATIONAL RESEARCH
Roland Molontay, Marcell Nagy
{"title":"How to improve the predictive validity of a composite admission score? A case study from Hungary","authors":"Roland Molontay, Marcell Nagy","doi":"10.1080/02602938.2022.2093835","DOIUrl":null,"url":null,"abstract":"Abstract An essential task in higher education is to construct a fair admission procedure. A great deal of research has been conducted on a central aspect of admission: predictive validity. However, to the best of our knowledge, this is the first study that investigates how the predictive validity of a composite admission score could be improved without redesigning the tests and introducing new measures. In this study, relying on the existing instruments of the Hungarian nationally standardized university entrance score, we construct an alternative score that not only has higher predictive validity but also a lower variation across disciplines and a smaller under- and overprediction bias in various student groups. To measure the predictive validity, we use an advanced statistical framework. The analysis relies on data of 24,675 students enrolled in the undergraduate programs of the Budapest University of Technology and Economics. We find that while the current score is effective in predicting university success, its predictive validity can be improved by a few changes: lifting the branching nature of the admission, focusing on general rather than program-specific knowledge, and introducing a multiplicative rewarding scheme for advanced level examinations.","PeriodicalId":48267,"journal":{"name":"Assessment & Evaluation in Higher Education","volume":null,"pages":null},"PeriodicalIF":4.1000,"publicationDate":"2022-07-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Assessment & Evaluation in Higher Education","FirstCategoryId":"95","ListUrlMain":"https://doi.org/10.1080/02602938.2022.2093835","RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}
引用次数: 1

Abstract

Abstract An essential task in higher education is to construct a fair admission procedure. A great deal of research has been conducted on a central aspect of admission: predictive validity. However, to the best of our knowledge, this is the first study that investigates how the predictive validity of a composite admission score could be improved without redesigning the tests and introducing new measures. In this study, relying on the existing instruments of the Hungarian nationally standardized university entrance score, we construct an alternative score that not only has higher predictive validity but also a lower variation across disciplines and a smaller under- and overprediction bias in various student groups. To measure the predictive validity, we use an advanced statistical framework. The analysis relies on data of 24,675 students enrolled in the undergraduate programs of the Budapest University of Technology and Economics. We find that while the current score is effective in predicting university success, its predictive validity can be improved by a few changes: lifting the branching nature of the admission, focusing on general rather than program-specific knowledge, and introducing a multiplicative rewarding scheme for advanced level examinations.
如何提高综合录取分数的预测有效性?匈牙利案例研究
摘要构建公平的录取程序是高等教育的一项重要任务。大量的研究已经进行了一个中心方面的承认:预测效度。然而,据我们所知,这是第一个研究如何在不重新设计测试和引入新措施的情况下提高综合录取分数的预测有效性的研究。在本研究中,依托匈牙利国家标准化大学入学分数的现有工具,我们构建了一个替代分数,该分数不仅具有较高的预测效度,而且在不同学生群体中具有较低的跨学科差异和较小的预测不足和高估偏差。为了测量预测的有效性,我们使用了一个先进的统计框架。该分析基于布达佩斯科技经济大学24,675名本科生的数据。我们发现,虽然目前的分数在预测大学成功方面是有效的,但它的预测有效性可以通过一些改变来提高:取消录取的分支性质,关注一般而不是特定项目的知识,并为高级水平考试引入乘法奖励方案。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Assessment & Evaluation in Higher Education
Assessment & Evaluation in Higher Education EDUCATION & EDUCATIONAL RESEARCH-
CiteScore
11.20
自引率
15.90%
发文量
70
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信