Limitations of the Inter-Unit Reliability: A Set of Practical Examples.

IF 1.6 Q3 HEALTH CARE SCIENCES & SERVICES
Nicholas Hartman, Vahakn B Shahinian, Valarie B Ashby, Katrina J Price, Kevin He
{"title":"Limitations of the Inter-Unit Reliability: A Set of Practical Examples.","authors":"Nicholas Hartman, Vahakn B Shahinian, Valarie B Ashby, Katrina J Price, Kevin He","doi":"10.1007/s10742-023-00307-0","DOIUrl":null,"url":null,"abstract":"<p><p>Healthcare quality measures are statistics that serve to evaluate healthcare providers and identify those that need to improve their care. Before using these measures in clinical practice, developers and reviewers assess measure reliability, which describes the degree to which differences in the measure values reflect actual variation in healthcare quality, as opposed to random noise. The Inter-Unit Reliability (IUR) is a popular statistic for assessing reliability, and it describes the proportion of total variation in a measure that is attributable to between-provider variation. However, Kalbfleisch, He, Xia, and Li (2018) [<i>Health Services and Outcomes Research Methodology</i>, 18, 215-225] have argued that the IUR has a severe limitation in that some of the between-provider variation may be unrelated to quality of care. In this paper, we illustrate the practical implications of this limitation through several concrete examples. We show that certain best-practices in measure development, such as careful risk adjustment and exclusion of unstable measure values, can decrease the sample IUR value. These findings uncover potential negative consequences of discarding measures with IUR values below some arbitrary threshold.</p>","PeriodicalId":45600,"journal":{"name":"Health Services and Outcomes Research Methodology","volume":null,"pages":null},"PeriodicalIF":1.6000,"publicationDate":"2024-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11323040/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Health Services and Outcomes Research Methodology","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1007/s10742-023-00307-0","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2023/6/28 0:00:00","PubModel":"Epub","JCR":"Q3","JCRName":"HEALTH CARE SCIENCES & SERVICES","Score":null,"Total":0}
引用次数: 0

Abstract

Healthcare quality measures are statistics that serve to evaluate healthcare providers and identify those that need to improve their care. Before using these measures in clinical practice, developers and reviewers assess measure reliability, which describes the degree to which differences in the measure values reflect actual variation in healthcare quality, as opposed to random noise. The Inter-Unit Reliability (IUR) is a popular statistic for assessing reliability, and it describes the proportion of total variation in a measure that is attributable to between-provider variation. However, Kalbfleisch, He, Xia, and Li (2018) [Health Services and Outcomes Research Methodology, 18, 215-225] have argued that the IUR has a severe limitation in that some of the between-provider variation may be unrelated to quality of care. In this paper, we illustrate the practical implications of this limitation through several concrete examples. We show that certain best-practices in measure development, such as careful risk adjustment and exclusion of unstable measure values, can decrease the sample IUR value. These findings uncover potential negative consequences of discarding measures with IUR values below some arbitrary threshold.

单位间可靠性的局限性:一组实际案例。
医疗质量测量是一种统计方法,用于评估医疗服务提供者,并找出需要改进的医疗服务提供者。在临床实践中使用这些测量指标之前,开发人员和审核人员要对测量指标的可靠性进行评估,可靠性描述了测量值的差异在多大程度上反映了医疗质量的实际变化,而不是随机噪音。单位间信度(IUR)是评估信度的常用统计量,它描述了医疗服务提供者之间的差异在测量值总差异中所占的比例。然而,Kalbfleisch、He、Xia 和 Li(2018)[《医疗服务与结果研究方法》,18,215-225] 认为,IUR 有一个严重的局限性,即部分医疗服务提供者之间的变异可能与医疗质量无关。在本文中,我们通过几个具体的例子来说明这一局限性的实际影响。我们表明,制定衡量标准时的某些最佳做法(如谨慎的风险调整和排除不稳定的衡量值)可以降低样本 IUR 值。这些发现揭示了剔除IUR值低于某个任意阈值的测量值可能带来的负面影响。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Health Services and Outcomes Research Methodology
Health Services and Outcomes Research Methodology HEALTH CARE SCIENCES & SERVICES-
CiteScore
3.40
自引率
6.70%
发文量
28
期刊介绍: The journal reflects the multidisciplinary nature of the field of health services and outcomes research. It addresses the needs of multiple, interlocking communities, including methodologists in statistics, econometrics, social and behavioral sciences; designers and analysts of health policy and health services research projects; and health care providers and policy makers who need to properly understand and evaluate the results of published research. The journal strives to enhance the level of methodologic rigor in health services and outcomes research and contributes to the development of methodologic standards in the field. In pursuing its main objective, the journal also provides a meeting ground for researchers from a number of traditional disciplines and fosters the development of new quantitative, qualitative, and mixed methods by statisticians, econometricians, health services researchers, and methodologists in other fields. Health Services and Outcomes Research Methodology publishes: Research papers on quantitative, qualitative, and mixed methods; Case Studies describing applications of quantitative and qualitative methodology in health services and outcomes research; Review Articles synthesizing and popularizing methodologic developments; Tutorials; Articles on computational issues and software reviews; Book reviews; and Notices. Special issues will be devoted to papers presented at important workshops and conferences.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信