以Goodhart法则为例:Meta-Gaming, Meta-Gaming和黑客学业表现指标

J. Griesemer
{"title":"以Goodhart法则为例:Meta-Gaming, Meta-Gaming和黑客学业表现指标","authors":"J. Griesemer","doi":"10.7551/mitpress/11087.003.0007","DOIUrl":null,"url":null,"abstract":"demic performance metrics and misconduct. One is that Goodhart’s law (1975) concerns more than simply the idea of individual responsiveness to pressures from societal policies, for example, central bank monetary policies employ economic performance measures as standards of regulation and control in banking. The other concerns how we might exploit what more there is to Goodhart’s law to probe the character of “mis”conduct, as individuals and organizations adapt to, and comply with, academic performance metrics institutionalized as standards. Contrast this with “bad” conduct, as individuals and organizations cynically attempt to “game” or “exploit” the system to achieve a better evaluation than their performance warrants. Along with other chapters in this volume (Csiszar, this volume, chapter 1; Power, this volume, chapter 3), I suggest Goodhart’s law describes conditions that not only undermine the representational success in modeling causal order in human social systems, but also the operation of the law inverts the causal order. Conversions of metrics into standards not only invite “gaming the system,” they also practically construct “gaming” as the new form of practice, rendering the original product or practice to be measured as a “side effect” in the new causal order. Or, as Wouters (this volume, chapter 4) urges, we must distinguish gaming the system from properly functioning in an inverted system. It is thus problematic to moralize and shame socalled “predatory” practices as if it were clear what constitutes ethical, nonpredatory practice in social worlds where Goodhart’s law operates. Goodhart’s lesson was that such measures are selfdefeating because they invite “mis”conduct. If people respond to standards as intended, the measure ceases to represent and record the primary target performance and comes to measure only compliance or conformity to the standard. The critique cuts deeper. As Lucas’s (1976) critique of macroeconometric 5 Taking Goodhart’s Law Meta: Gaming, MetaGaming, and Hacking Academic Performance Metrics","PeriodicalId":186262,"journal":{"name":"Gaming the Metrics","volume":"129 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"2","resultStr":"{\"title\":\"Taking Goodhart’s Law Meta: Gaming, Meta-Gaming, and Hacking Academic Performance Metrics\",\"authors\":\"J. Griesemer\",\"doi\":\"10.7551/mitpress/11087.003.0007\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"demic performance metrics and misconduct. One is that Goodhart’s law (1975) concerns more than simply the idea of individual responsiveness to pressures from societal policies, for example, central bank monetary policies employ economic performance measures as standards of regulation and control in banking. The other concerns how we might exploit what more there is to Goodhart’s law to probe the character of “mis”conduct, as individuals and organizations adapt to, and comply with, academic performance metrics institutionalized as standards. Contrast this with “bad” conduct, as individuals and organizations cynically attempt to “game” or “exploit” the system to achieve a better evaluation than their performance warrants. Along with other chapters in this volume (Csiszar, this volume, chapter 1; Power, this volume, chapter 3), I suggest Goodhart’s law describes conditions that not only undermine the representational success in modeling causal order in human social systems, but also the operation of the law inverts the causal order. Conversions of metrics into standards not only invite “gaming the system,” they also practically construct “gaming” as the new form of practice, rendering the original product or practice to be measured as a “side effect” in the new causal order. Or, as Wouters (this volume, chapter 4) urges, we must distinguish gaming the system from properly functioning in an inverted system. It is thus problematic to moralize and shame socalled “predatory” practices as if it were clear what constitutes ethical, nonpredatory practice in social worlds where Goodhart’s law operates. Goodhart’s lesson was that such measures are selfdefeating because they invite “mis”conduct. If people respond to standards as intended, the measure ceases to represent and record the primary target performance and comes to measure only compliance or conformity to the standard. The critique cuts deeper. As Lucas’s (1976) critique of macroeconometric 5 Taking Goodhart’s Law Meta: Gaming, MetaGaming, and Hacking Academic Performance Metrics\",\"PeriodicalId\":186262,\"journal\":{\"name\":\"Gaming the Metrics\",\"volume\":\"129 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"1900-01-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"2\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Gaming the Metrics\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.7551/mitpress/11087.003.0007\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Gaming the Metrics","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.7551/mitpress/11087.003.0007","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 2

摘要

学术绩效指标和不当行为。一是古德哈特定律(1975)不仅仅关注个人对社会政策压力的反应,例如,中央银行的货币政策采用经济表现指标作为银行监管和控制的标准。另一个问题是,当个人和组织适应并遵守作为标准的学术表现指标时,我们如何利用古德哈特定律之外的更多内容来探究“不当”行为的特征。这与“坏”行为形成对比,因为个人和组织愤世嫉俗地试图“玩弄”或“利用”系统,以获得比他们的表现所保证的更好的评估。以及本卷的其他章节(cisszar,本卷,第一章;权力,本卷,第3章),我认为古德哈特定律描述的条件不仅破坏了人类社会系统中因果秩序建模的代表性成功,而且该定律的运作颠倒了因果秩序。将参数转换为标准不仅会导致“在系统中玩游戏”,而且还会将“游戏”作为一种新的实践形式,将原始产品或实践呈现为新的因果秩序中的“副作用”。或者,正如Wouters(本卷第4章)所敦促的那样,我们必须区分博弈系统与在反向系统中正常运行。因此,把所谓的“掠夺性”行为道德化和羞辱是有问题的,就好像在古德哈特法则起作用的社会世界里,什么是道德的、非掠夺性的行为是很清楚的一样。古德哈特的教训是,这些措施会弄巧成拙,因为它们会招致“不当”行为。如果人们按照预期对标准作出反应,则度量就不再代表和记录主要目标性能,而只度量对标准的遵从性或一致性。这种批评更加深刻。正如Lucas(1976)对宏观计量经济学的批判5采用Goodhart法则:Meta: Gaming, MetaGaming和Hacking Academic Performance Metrics
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Taking Goodhart’s Law Meta: Gaming, Meta-Gaming, and Hacking Academic Performance Metrics
demic performance metrics and misconduct. One is that Goodhart’s law (1975) concerns more than simply the idea of individual responsiveness to pressures from societal policies, for example, central bank monetary policies employ economic performance measures as standards of regulation and control in banking. The other concerns how we might exploit what more there is to Goodhart’s law to probe the character of “mis”conduct, as individuals and organizations adapt to, and comply with, academic performance metrics institutionalized as standards. Contrast this with “bad” conduct, as individuals and organizations cynically attempt to “game” or “exploit” the system to achieve a better evaluation than their performance warrants. Along with other chapters in this volume (Csiszar, this volume, chapter 1; Power, this volume, chapter 3), I suggest Goodhart’s law describes conditions that not only undermine the representational success in modeling causal order in human social systems, but also the operation of the law inverts the causal order. Conversions of metrics into standards not only invite “gaming the system,” they also practically construct “gaming” as the new form of practice, rendering the original product or practice to be measured as a “side effect” in the new causal order. Or, as Wouters (this volume, chapter 4) urges, we must distinguish gaming the system from properly functioning in an inverted system. It is thus problematic to moralize and shame socalled “predatory” practices as if it were clear what constitutes ethical, nonpredatory practice in social worlds where Goodhart’s law operates. Goodhart’s lesson was that such measures are selfdefeating because they invite “mis”conduct. If people respond to standards as intended, the measure ceases to represent and record the primary target performance and comes to measure only compliance or conformity to the standard. The critique cuts deeper. As Lucas’s (1976) critique of macroeconometric 5 Taking Goodhart’s Law Meta: Gaming, MetaGaming, and Hacking Academic Performance Metrics
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信