好、坏、丑:当我们找出最好和最差的组织时,我们到底在做什么?

IF 5.6 1区 医学 Q1 HEALTH CARE SCIENCES & SERVICES
Gary A Abel, Denis Agniel, Marc N Elliott
{"title":"好、坏、丑:当我们找出最好和最差的组织时,我们到底在做什么?","authors":"Gary A Abel, Denis Agniel, Marc N Elliott","doi":"10.1136/bmjqs-2023-017039","DOIUrl":null,"url":null,"abstract":"<p><p>Identifying high and poorly performing organisations is common practice in healthcare. Often this is done within a frequentist inferential framework where statistical techniques are used that acknowledge that observed performance is an imperfect measure of underlying quality. Various methods are employed for this purpose, but the influence of chance on the degree of misclassification is often underappreciated. Using simulations, we show that the distribution of underlying performance of organisations flagged as the worst performers, using current best practices, was highly dependent on the reliability of the performance measure. When reliability was low, flagged organisations were likely to have an underlying performance that was near the population average. Reliability needs to reach at least 0.7 for 50% of flagged organisations to be correctly flagged and 0.9 to nearly eliminate incorrectly flagging organisations close to the overall mean. We conclude that despite their widespread use, techniques for identifying the best and worst performing organisations do not necessarily identify truly good and bad performers and even with the best techniques, reliable data are required.</p>","PeriodicalId":9077,"journal":{"name":"BMJ Quality & Safety","volume":null,"pages":null},"PeriodicalIF":5.6000,"publicationDate":"2024-10-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"The good, the bad and the ugly: What do we really do when we identify the best and the worst organisations?\",\"authors\":\"Gary A Abel, Denis Agniel, Marc N Elliott\",\"doi\":\"10.1136/bmjqs-2023-017039\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><p>Identifying high and poorly performing organisations is common practice in healthcare. Often this is done within a frequentist inferential framework where statistical techniques are used that acknowledge that observed performance is an imperfect measure of underlying quality. Various methods are employed for this purpose, but the influence of chance on the degree of misclassification is often underappreciated. Using simulations, we show that the distribution of underlying performance of organisations flagged as the worst performers, using current best practices, was highly dependent on the reliability of the performance measure. When reliability was low, flagged organisations were likely to have an underlying performance that was near the population average. Reliability needs to reach at least 0.7 for 50% of flagged organisations to be correctly flagged and 0.9 to nearly eliminate incorrectly flagging organisations close to the overall mean. We conclude that despite their widespread use, techniques for identifying the best and worst performing organisations do not necessarily identify truly good and bad performers and even with the best techniques, reliable data are required.</p>\",\"PeriodicalId\":9077,\"journal\":{\"name\":\"BMJ Quality & Safety\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":5.6000,\"publicationDate\":\"2024-10-18\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"BMJ Quality & Safety\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1136/bmjqs-2023-017039\",\"RegionNum\":1,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"HEALTH CARE SCIENCES & SERVICES\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"BMJ Quality & Safety","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1136/bmjqs-2023-017039","RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"HEALTH CARE SCIENCES & SERVICES","Score":null,"Total":0}
引用次数: 0

摘要

识别绩效高和绩效差的机构是医疗保健行业的常见做法。这通常是在频数推论框架内进行的,其中使用的统计技术承认观察到的绩效并不能完全衡量基本质量。为此,我们采用了多种方法,但偶然性对误判程度的影响往往未得到充分重视。通过模拟,我们发现,使用当前的最佳实践,被标记为表现最差的组织的基本绩效分布在很大程度上取决于绩效衡量标准的可靠性。当可靠性较低时,被标记为最差的组织的基本绩效很可能接近总体平均水平。可靠度至少要达到 0.7,才能正确标示出 50%的被标示机构;可靠度达到 0.9,才能几乎消除接近总体平均值的错误标示机构。我们的结论是,尽管识别表现最佳和最差组织的技术被广泛使用,但并不一定能识别出真正的表现好和表现差的组织,而且即使使用最好的技术,也需要可靠的数据。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
The good, the bad and the ugly: What do we really do when we identify the best and the worst organisations?

Identifying high and poorly performing organisations is common practice in healthcare. Often this is done within a frequentist inferential framework where statistical techniques are used that acknowledge that observed performance is an imperfect measure of underlying quality. Various methods are employed for this purpose, but the influence of chance on the degree of misclassification is often underappreciated. Using simulations, we show that the distribution of underlying performance of organisations flagged as the worst performers, using current best practices, was highly dependent on the reliability of the performance measure. When reliability was low, flagged organisations were likely to have an underlying performance that was near the population average. Reliability needs to reach at least 0.7 for 50% of flagged organisations to be correctly flagged and 0.9 to nearly eliminate incorrectly flagging organisations close to the overall mean. We conclude that despite their widespread use, techniques for identifying the best and worst performing organisations do not necessarily identify truly good and bad performers and even with the best techniques, reliable data are required.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
BMJ Quality & Safety
BMJ Quality & Safety HEALTH CARE SCIENCES & SERVICES-
CiteScore
9.80
自引率
7.40%
发文量
104
审稿时长
4-8 weeks
期刊介绍: BMJ Quality & Safety (previously Quality & Safety in Health Care) is an international peer review publication providing research, opinions, debates and reviews for academics, clinicians and healthcare managers focused on the quality and safety of health care and the science of improvement. The journal receives approximately 1000 manuscripts a year and has an acceptance rate for original research of 12%. Time from submission to first decision averages 22 days and accepted articles are typically published online within 20 days. Its current impact factor is 3.281.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信