Judging the judges: evaluating the accuracy and national bias of international gymnastics judges

IF 1.1 Q3 SOCIAL SCIENCES, MATHEMATICAL METHODS
Sandro Heiniger, Hugues Mercier
{"title":"Judging the judges: evaluating the accuracy and national bias of international gymnastics judges","authors":"Sandro Heiniger, Hugues Mercier","doi":"10.1515/jqas-2019-0113","DOIUrl":null,"url":null,"abstract":"Abstract We design, describe and implement a statistical engine to analyze the performance of gymnastics judges with three objectives: (1) provide constructive feedback to judges, executive committees and national federations; (2) assign the best judges to the most important competitions; (3) detect bias and persistent misjudging. Judging a gymnastics routine is a random process, and we model this process using heteroscedastic random variables. The developed marking score scales the difference between the mark of a judge and the true performance level of a gymnast as a function of the intrinsic judging error variability estimated from historical data for each apparatus. This dependence between judging variability and performance quality has never been properly studied. We leverage the intrinsic judging error variability and the marking score to detect outlier marks and study the national bias of judges favoring athletes of the same nationality. We also study ranking scores assessing to what extent judges rate gymnasts in the correct order. Our main observation is that there are significant differences between the best and worst judges, both in terms of accuracy and national bias. The insights from this work have led to recommendations and rule changes at the Fédération Internationale de Gymnastique.","PeriodicalId":16925,"journal":{"name":"Journal of Quantitative Analysis in Sports","volume":null,"pages":null},"PeriodicalIF":1.1000,"publicationDate":"2021-10-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"9","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Quantitative Analysis in Sports","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1515/jqas-2019-0113","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"SOCIAL SCIENCES, MATHEMATICAL METHODS","Score":null,"Total":0}
引用次数: 9

Abstract

Abstract We design, describe and implement a statistical engine to analyze the performance of gymnastics judges with three objectives: (1) provide constructive feedback to judges, executive committees and national federations; (2) assign the best judges to the most important competitions; (3) detect bias and persistent misjudging. Judging a gymnastics routine is a random process, and we model this process using heteroscedastic random variables. The developed marking score scales the difference between the mark of a judge and the true performance level of a gymnast as a function of the intrinsic judging error variability estimated from historical data for each apparatus. This dependence between judging variability and performance quality has never been properly studied. We leverage the intrinsic judging error variability and the marking score to detect outlier marks and study the national bias of judges favoring athletes of the same nationality. We also study ranking scores assessing to what extent judges rate gymnasts in the correct order. Our main observation is that there are significant differences between the best and worst judges, both in terms of accuracy and national bias. The insights from this work have led to recommendations and rule changes at the Fédération Internationale de Gymnastique.
评判裁判:评价国际体操裁判的准确性和国家偏见
摘要:本文设计、描述并实现了一个统计引擎来分析体操裁判的表现,其目标有三个:(1)为裁判、执行委员会和国家联合会提供建设性的反馈;(2)选出最优秀的裁判参加最重要的比赛;(3)发现偏见和持续的误判。判断一个体操动作是一个随机过程,我们使用异方差随机变量对这个过程进行建模。所开发的评分分数将裁判的评分与体操运动员的真实表现水平之间的差异作为每个器械的历史数据估计的固有判断误差变异性的函数。判断可变性和性能质量之间的这种依赖关系从未得到过适当的研究。我们利用固有判断误差变异性和评分分数来检测异常值,研究裁判偏向同一国籍运动员的民族偏见。我们还研究了排名分数,以评估裁判在多大程度上以正确的顺序评价体操运动员。我们的主要观察是,在准确性和民族偏见方面,最好和最差的法官之间存在显著差异。这项工作的见解导致了国际体操联合会的建议和规则的变化。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Journal of Quantitative Analysis in Sports
Journal of Quantitative Analysis in Sports SOCIAL SCIENCES, MATHEMATICAL METHODS-
CiteScore
2.00
自引率
12.50%
发文量
15
期刊介绍: The Journal of Quantitative Analysis in Sports (JQAS), an official journal of the American Statistical Association, publishes timely, high-quality peer-reviewed research on the quantitative aspects of professional and amateur sports, including collegiate and Olympic competition. The scope of application reflects the increasing demand for novel methods to analyze and understand data in the growing field of sports analytics. Articles come from a wide variety of sports and diverse perspectives, and address topics such as game outcome models, measurement and evaluation of player performance, tournament structure, analysis of rules and adjudication, within-game strategy, analysis of sporting technologies, and player and team ranking methods. JQAS seeks to publish manuscripts that demonstrate original ways of approaching problems, develop cutting edge methods, and apply innovative thinking to solve difficult challenges in sports contexts. JQAS brings together researchers from various disciplines, including statistics, operations research, machine learning, scientific computing, econometrics, and sports management.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信