Limitations of the Inter-Unit Reliability: A Set of Practical Examples.

IF 1.3 4区 农林科学 Q2 VETERINARY SCIENCES
Avian Diseases Pub Date : 2024-06-01 Epub Date: 2023-06-28 DOI:10.1007/s10742-023-00307-0
Nicholas Hartman, Vahakn B Shahinian, Valarie B Ashby, Katrina J Price, Kevin He
{"title":"Limitations of the Inter-Unit Reliability: A Set of Practical Examples.","authors":"Nicholas Hartman, Vahakn B Shahinian, Valarie B Ashby, Katrina J Price, Kevin He","doi":"10.1007/s10742-023-00307-0","DOIUrl":null,"url":null,"abstract":"<p><p>Healthcare quality measures are statistics that serve to evaluate healthcare providers and identify those that need to improve their care. Before using these measures in clinical practice, developers and reviewers assess measure reliability, which describes the degree to which differences in the measure values reflect actual variation in healthcare quality, as opposed to random noise. The Inter-Unit Reliability (IUR) is a popular statistic for assessing reliability, and it describes the proportion of total variation in a measure that is attributable to between-provider variation. However, Kalbfleisch, He, Xia, and Li (2018) [<i>Health Services and Outcomes Research Methodology</i>, 18, 215-225] have argued that the IUR has a severe limitation in that some of the between-provider variation may be unrelated to quality of care. In this paper, we illustrate the practical implications of this limitation through several concrete examples. We show that certain best-practices in measure development, such as careful risk adjustment and exclusion of unstable measure values, can decrease the sample IUR value. These findings uncover potential negative consequences of discarding measures with IUR values below some arbitrary threshold.</p>","PeriodicalId":8667,"journal":{"name":"Avian Diseases","volume":"26 1 1","pages":"156-169"},"PeriodicalIF":1.3000,"publicationDate":"2024-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11323040/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Avian Diseases","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1007/s10742-023-00307-0","RegionNum":4,"RegionCategory":"农林科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2023/6/28 0:00:00","PubModel":"Epub","JCR":"Q2","JCRName":"VETERINARY SCIENCES","Score":null,"Total":0}
引用次数: 0

Abstract

Healthcare quality measures are statistics that serve to evaluate healthcare providers and identify those that need to improve their care. Before using these measures in clinical practice, developers and reviewers assess measure reliability, which describes the degree to which differences in the measure values reflect actual variation in healthcare quality, as opposed to random noise. The Inter-Unit Reliability (IUR) is a popular statistic for assessing reliability, and it describes the proportion of total variation in a measure that is attributable to between-provider variation. However, Kalbfleisch, He, Xia, and Li (2018) [Health Services and Outcomes Research Methodology, 18, 215-225] have argued that the IUR has a severe limitation in that some of the between-provider variation may be unrelated to quality of care. In this paper, we illustrate the practical implications of this limitation through several concrete examples. We show that certain best-practices in measure development, such as careful risk adjustment and exclusion of unstable measure values, can decrease the sample IUR value. These findings uncover potential negative consequences of discarding measures with IUR values below some arbitrary threshold.

单位间可靠性的局限性:一组实际案例。
医疗质量测量是一种统计方法,用于评估医疗服务提供者,并找出需要改进的医疗服务提供者。在临床实践中使用这些测量指标之前,开发人员和审核人员要对测量指标的可靠性进行评估,可靠性描述了测量值的差异在多大程度上反映了医疗质量的实际变化,而不是随机噪音。单位间信度(IUR)是评估信度的常用统计量,它描述了医疗服务提供者之间的差异在测量值总差异中所占的比例。然而,Kalbfleisch、He、Xia 和 Li(2018)[《医疗服务与结果研究方法》,18,215-225] 认为,IUR 有一个严重的局限性,即部分医疗服务提供者之间的变异可能与医疗质量无关。在本文中,我们通过几个具体的例子来说明这一局限性的实际影响。我们表明,制定衡量标准时的某些最佳做法(如谨慎的风险调整和排除不稳定的衡量值)可以降低样本 IUR 值。这些发现揭示了剔除IUR值低于某个任意阈值的测量值可能带来的负面影响。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Avian Diseases
Avian Diseases 农林科学-兽医学
CiteScore
2.40
自引率
7.10%
发文量
80
审稿时长
3 months
期刊介绍: Avian Diseases is an international journal dedicated to publishing original basic or clinical research of the highest quality from various disciplines including microbiology, immunology, pathology and epidemiology. Papers on avian diseases relevant to etiology, pathogenesis, diagnosis, treatment, and control are accepted. Manuscripts dealing with avian species other than poultry will be considered only if the subject is relevant to poultry health.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信