Nicholas Hartman, Vahakn B Shahinian, Valarie B Ashby, Katrina J Price, Kevin He
{"title":"Limitations of the Inter-Unit Reliability: A Set of Practical Examples.","authors":"Nicholas Hartman, Vahakn B Shahinian, Valarie B Ashby, Katrina J Price, Kevin He","doi":"10.1007/s10742-023-00307-0","DOIUrl":null,"url":null,"abstract":"<p><p>Healthcare quality measures are statistics that serve to evaluate healthcare providers and identify those that need to improve their care. Before using these measures in clinical practice, developers and reviewers assess measure reliability, which describes the degree to which differences in the measure values reflect actual variation in healthcare quality, as opposed to random noise. The Inter-Unit Reliability (IUR) is a popular statistic for assessing reliability, and it describes the proportion of total variation in a measure that is attributable to between-provider variation. However, Kalbfleisch, He, Xia, and Li (2018) [<i>Health Services and Outcomes Research Methodology</i>, 18, 215-225] have argued that the IUR has a severe limitation in that some of the between-provider variation may be unrelated to quality of care. In this paper, we illustrate the practical implications of this limitation through several concrete examples. We show that certain best-practices in measure development, such as careful risk adjustment and exclusion of unstable measure values, can decrease the sample IUR value. These findings uncover potential negative consequences of discarding measures with IUR values below some arbitrary threshold.</p>","PeriodicalId":8667,"journal":{"name":"Avian Diseases","volume":"26 1 1","pages":"156-169"},"PeriodicalIF":1.3000,"publicationDate":"2024-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11323040/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Avian Diseases","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1007/s10742-023-00307-0","RegionNum":4,"RegionCategory":"农林科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2023/6/28 0:00:00","PubModel":"Epub","JCR":"Q2","JCRName":"VETERINARY SCIENCES","Score":null,"Total":0}
引用次数: 0
Abstract
Healthcare quality measures are statistics that serve to evaluate healthcare providers and identify those that need to improve their care. Before using these measures in clinical practice, developers and reviewers assess measure reliability, which describes the degree to which differences in the measure values reflect actual variation in healthcare quality, as opposed to random noise. The Inter-Unit Reliability (IUR) is a popular statistic for assessing reliability, and it describes the proportion of total variation in a measure that is attributable to between-provider variation. However, Kalbfleisch, He, Xia, and Li (2018) [Health Services and Outcomes Research Methodology, 18, 215-225] have argued that the IUR has a severe limitation in that some of the between-provider variation may be unrelated to quality of care. In this paper, we illustrate the practical implications of this limitation through several concrete examples. We show that certain best-practices in measure development, such as careful risk adjustment and exclusion of unstable measure values, can decrease the sample IUR value. These findings uncover potential negative consequences of discarding measures with IUR values below some arbitrary threshold.
期刊介绍:
Avian Diseases is an international journal dedicated to publishing original basic or clinical research of the highest quality from various disciplines including microbiology, immunology, pathology and epidemiology. Papers on avian diseases relevant to etiology, pathogenesis, diagnosis, treatment, and control are accepted. Manuscripts dealing with avian species other than poultry will be considered only if the subject is relevant to poultry health.