Interrater agreement in classifying infections during extracorporeal membrane oxygenation.

IF 1.4 4区 医学 Q4 ENGINEERING, BIOMEDICAL
Karlijn Verkerk, Lara Ca Pladet, Christiaan L Meuwese, Dirk W Donker, Lennie Pg Derde, Olaf L Cremer
{"title":"Interrater agreement in classifying infections during extracorporeal membrane oxygenation.","authors":"Karlijn Verkerk, Lara Ca Pladet, Christiaan L Meuwese, Dirk W Donker, Lennie Pg Derde, Olaf L Cremer","doi":"10.1177/03913988231193448","DOIUrl":null,"url":null,"abstract":"<p><p>Infectious complications are common during extracorporeal membrane oxygenation (ECMO) and may negatively impact outcomes. However, there is considerable variation in the reported rates of incidence, which hampers the use of infections as a quality benchmark for ECMO centers. To assess the contributing role of poor interrater agreement, three independent raters reviewed medical records from all intensive care unit (ICU) patients who received ECMO for >24 h in our tertiary center between October 2019 and October 2021 for suspected episodes of infection, which were rated based on their date of onset and presumed site/diagnosis. To establish a gold standard, any discrepancies were resolved using an expert panel consisting of two intensivists/infectious disease specialists. During 83 ECMO-runs in 77 patients, we observed a total of 62 adjudicated infectious episodes (incidence rate 62, 95% CI: 48-80, per 1000 days at risk). Among 81 episodes suspected by at least one observer, 66 (81%) were identified by two, and only 44 (54%) by all three raters, resulting in Fleiss' kappa of 0.10 (95% CI: 0.00-0.19; slight agreement). However, if raters concurred regarding infection onset, subsequent agreement on infection site was good (concordance 89%; kappa 0.85, 95% CI: 0.72-0.98; near perfect agreement). In conclusion, adjudication of infectious episodes during ECMO is associated with poor interrater agreement regarding occurrence-but not site-of infection. This finding might partially explain the significant disparities observed in reported infection rates during ECMO, emphasizing the need for caution when interpreting infection data in this particular population due to the potential for inherent measurement error.</p>","PeriodicalId":13932,"journal":{"name":"International Journal of Artificial Organs","volume":null,"pages":null},"PeriodicalIF":1.4000,"publicationDate":"2023-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10629257/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Journal of Artificial Organs","FirstCategoryId":"5","ListUrlMain":"https://doi.org/10.1177/03913988231193448","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2023/8/19 0:00:00","PubModel":"Epub","JCR":"Q4","JCRName":"ENGINEERING, BIOMEDICAL","Score":null,"Total":0}
引用次数: 0

Abstract

Infectious complications are common during extracorporeal membrane oxygenation (ECMO) and may negatively impact outcomes. However, there is considerable variation in the reported rates of incidence, which hampers the use of infections as a quality benchmark for ECMO centers. To assess the contributing role of poor interrater agreement, three independent raters reviewed medical records from all intensive care unit (ICU) patients who received ECMO for >24 h in our tertiary center between October 2019 and October 2021 for suspected episodes of infection, which were rated based on their date of onset and presumed site/diagnosis. To establish a gold standard, any discrepancies were resolved using an expert panel consisting of two intensivists/infectious disease specialists. During 83 ECMO-runs in 77 patients, we observed a total of 62 adjudicated infectious episodes (incidence rate 62, 95% CI: 48-80, per 1000 days at risk). Among 81 episodes suspected by at least one observer, 66 (81%) were identified by two, and only 44 (54%) by all three raters, resulting in Fleiss' kappa of 0.10 (95% CI: 0.00-0.19; slight agreement). However, if raters concurred regarding infection onset, subsequent agreement on infection site was good (concordance 89%; kappa 0.85, 95% CI: 0.72-0.98; near perfect agreement). In conclusion, adjudication of infectious episodes during ECMO is associated with poor interrater agreement regarding occurrence-but not site-of infection. This finding might partially explain the significant disparities observed in reported infection rates during ECMO, emphasizing the need for caution when interpreting infection data in this particular population due to the potential for inherent measurement error.

Abstract Image

Abstract Image

Abstract Image

体外膜氧合过程中感染分类的一致性。
感染性并发症在体外膜肺氧合(ECMO)期间很常见,可能会对结果产生负面影响。然而,报告的发病率存在相当大的差异,这阻碍了将感染作为ECMO中心的质量基准。为了评估参与者之间不一致的影响,三名独立评分者审查了所有接受ECMO治疗>24的重症监护室(ICU)患者的医疗记录 h在2019年10月至2021年10月期间在我们的三级中心进行疑似感染,根据其发病日期和推测的部位/诊断进行评分。为了建立一个金标准,任何差异都通过由两名重症医生/传染病专家组成的专家小组来解决。在77名患者的83次ECMO运行过程中,我们总共观察到62次判定的感染发作(发病率62,95%CI:48-80,/1000 风险天数)。在至少一名观察者怀疑的81例发作中,66例(81%)由两名观察者确定,三名评分者仅确定44例(54%),Fleiss’kappa为0.10(95%可信区间:0.00-0.19;略有一致)。然而,如果评分者对感染发生有一致意见,那么随后对感染部位的一致性良好(一致性89%;kappa 0.85,95%CI:0.72-0.98;接近完全一致)。总之,ECMO期间感染事件的判断与患者之间对感染发生的一致性差有关,但与感染部位无关。这一发现可能部分解释了在ECMO期间观察到的报告感染率的显著差异,强调在解释这一特定人群的感染数据时需要谨慎,因为可能存在固有的测量误差。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
International Journal of Artificial Organs
International Journal of Artificial Organs 医学-工程:生物医学
CiteScore
3.40
自引率
5.90%
发文量
92
审稿时长
3 months
期刊介绍: The International Journal of Artificial Organs (IJAO) publishes peer-reviewed research and clinical, experimental and theoretical, contributions to the field of artificial, bioartificial and tissue-engineered organs. The mission of the IJAO is to foster the development and optimization of artificial, bioartificial and tissue-engineered organs, for implantation or use in procedures, to treat functional deficits of all human tissues and organs.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信