Hedging-based scoring rules for multiple-choice questions

IF 2.3 3区经济学 Q2 ECONOMICS

Journal of Economic Behavior & Organization Pub Date : 2025-08-11 DOI:10.1016/j.jebo.2025.107184

Jingcheng Fu , Xing Zhang , Songfa Zhong

{"title":"Hedging-based scoring rules for multiple-choice questions","authors":"Jingcheng Fu , Xing Zhang , Songfa Zhong","doi":"10.1016/j.jebo.2025.107184","DOIUrl":null,"url":null,"abstract":"<div><div>This paper proposes two novel scoring rules for multiple-choice questions based on the test-takers’ propensity to hedge across possible answers. To examine these scoring rules, we randomly assign 2,986 participants in an IQ test into three conditions. In the control condition, participants choose one option, and receive one point for a correct response. In the treatment conditions, they can explicitly hedge by choosing k options: if the correct option is among the k chosen options, they receive 1/k point in the outcome-mixing treatment, and one point with probability 1/k in the probability-mixing treatment. We find that participants in both treatments hedge pervasively, score lower compared to those in the control, but do not improve their time allocation. While scores in the three conditions exhibit similar psychometric quality, we observe a significant correlation between academic performance and IQ score measured in the probability-mixing condition. These results highlight the potential of these two hedging-based scoring rules for practical applications in multiple-choice questions.</div></div>","PeriodicalId":48409,"journal":{"name":"Journal of Economic Behavior & Organization","volume":"237 ","pages":"Article 107184"},"PeriodicalIF":2.3000,"publicationDate":"2025-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Economic Behavior & Organization","FirstCategoryId":"96","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0167268125003038","RegionNum":3,"RegionCategory":"经济学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ECONOMICS","Score":null,"Total":0}

引用次数: 0

Abstract

This paper proposes two novel scoring rules for multiple-choice questions based on the test-takers’ propensity to hedge across possible answers. To examine these scoring rules, we randomly assign 2,986 participants in an IQ test into three conditions. In the control condition, participants choose one option, and receive one point for a correct response. In the treatment conditions, they can explicitly hedge by choosing k options: if the correct option is among the k chosen options, they receive 1/k point in the outcome-mixing treatment, and one point with probability 1/k in the probability-mixing treatment. We find that participants in both treatments hedge pervasively, score lower compared to those in the control, but do not improve their time allocation. While scores in the three conditions exhibit similar psychometric quality, we observe a significant correlation between academic performance and IQ score measured in the probability-mixing condition. These results highlight the potential of these two hedging-based scoring rules for practical applications in multiple-choice questions.

查看原文本刊更多论文

多项选择题的对冲计分规则

本文提出了两种新的多项选择题评分规则，该评分规则基于考生对可能答案的对冲倾向。为了检验这些评分规则，我们将2986名参与者随机分配到三种情况下进行智商测试。在控制条件下，参与者选择一个选项，正确回答得一分。在处理条件下，他们可以通过选择k个选项来显式对冲：如果正确的选项在k个选择选项中，他们在结果混合处理中得到1/k点，在概率混合处理中得到1/k点。我们发现，两种治疗的参与者普遍回避，得分低于对照组，但没有改善他们的时间分配。虽然这三种条件下的得分表现出相似的心理测量质量，但我们观察到在概率混合条件下的学业成绩和智商得分之间存在显著的相关性。这些结果突出了这两个基于对冲的评分规则在多项选择题中的实际应用的潜力。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of Economic Behavior & Organization ECONOMICS-

CiteScore

3.80

自引率

9.10%

发文量

392

期刊介绍： The Journal of Economic Behavior and Organization is devoted to theoretical and empirical research concerning economic decision, organization and behavior and to economic change in all its aspects. Its specific purposes are to foster an improved understanding of how human cognitive, computational and informational characteristics influence the working of economic organizations and market economies and how an economy structural features lead to various types of micro and macro behavior, to changing patterns of development and to institutional evolution. Research with these purposes that explore the interrelations of economics with other disciplines such as biology, psychology, law, anthropology, sociology and mathematics is particularly welcome.