An interrater reliability analysis of preoperative mortality risk calculators used for elective high-risk noncardiac surgical patients shows poor to moderate reliability.

IF 2.3 3区 医学 Q2 ANESTHESIOLOGY
Jacqueline E M Vernooij, Lian Roovers, René van der Zwan, Benedikt Preckel, Cor J Kalkman, Nick J Koning
{"title":"An interrater reliability analysis of preoperative mortality risk calculators used for elective high-risk noncardiac surgical patients shows poor to moderate reliability.","authors":"Jacqueline E M Vernooij, Lian Roovers, René van der Zwan, Benedikt Preckel, Cor J Kalkman, Nick J Koning","doi":"10.1186/s12871-024-02771-8","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>Multiple preoperative calculators are available online to predict preoperative mortality risk for noncardiac surgical patients. However, it is currently unknown how these risk calculators perform across different raters. The current study investigated the interrater reliability of three preoperative mortality risk calculators in an elective high-risk noncardiac surgical patient population to evaluate if these calculators can be safely used for identification of high-risk noncardiac surgical patients for a preoperative multidisciplinary team discussion.</p><p><strong>Methods: </strong>Five anesthesiologists assessed the preoperative mortality risk of 34 high-risk patients using the preoperative score to calculate postoperative mortality risks (POSPOM), the American College of Surgeons surgical risk calculator (SRC), and the surgical outcome risk tool (SORT). In total, 170 calculations per calculator were gathered.</p><p><strong>Results: </strong>Interrater reliability was poor for SORT (ICC (C.I. 95%) = 0.46 (0.30-0.63)) and moderate for SRC (ICC = 0.65 (0.51-0.78)) and POSPOM (ICC = 0.63 (0.49-0.77). The absolute range of calculated mortality risk was 0.2-72% for POSPOM, 0-36% for SRC, and 0.4-17% for SORT. The coefficient of variation increased in higher risk classes for POSPOM and SORT. The extended Bland-Altman limits of agreement suggested that all raters contributed to the variation in calculated risks.</p><p><strong>Conclusion: </strong>The current results indicate that the preoperative risk calculators POSPOM, SRC, and SORT exhibit poor to moderate interrater reliability. These calculators are not sufficiently accurate for clinical identification and preoperative counseling of high-risk surgical patients. Clinicians should be trained in using mortality risk calculators. Also, clinicians should be cautious when using predicted mortality estimates from these calculators to identify high-risk noncardiac surgical patients for elective surgery.</p>","PeriodicalId":9190,"journal":{"name":"BMC Anesthesiology","volume":"24 1","pages":"392"},"PeriodicalIF":2.3000,"publicationDate":"2024-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11523836/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"BMC Anesthesiology","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1186/s12871-024-02771-8","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ANESTHESIOLOGY","Score":null,"Total":0}
引用次数: 0

Abstract

Background: Multiple preoperative calculators are available online to predict preoperative mortality risk for noncardiac surgical patients. However, it is currently unknown how these risk calculators perform across different raters. The current study investigated the interrater reliability of three preoperative mortality risk calculators in an elective high-risk noncardiac surgical patient population to evaluate if these calculators can be safely used for identification of high-risk noncardiac surgical patients for a preoperative multidisciplinary team discussion.

Methods: Five anesthesiologists assessed the preoperative mortality risk of 34 high-risk patients using the preoperative score to calculate postoperative mortality risks (POSPOM), the American College of Surgeons surgical risk calculator (SRC), and the surgical outcome risk tool (SORT). In total, 170 calculations per calculator were gathered.

Results: Interrater reliability was poor for SORT (ICC (C.I. 95%) = 0.46 (0.30-0.63)) and moderate for SRC (ICC = 0.65 (0.51-0.78)) and POSPOM (ICC = 0.63 (0.49-0.77). The absolute range of calculated mortality risk was 0.2-72% for POSPOM, 0-36% for SRC, and 0.4-17% for SORT. The coefficient of variation increased in higher risk classes for POSPOM and SORT. The extended Bland-Altman limits of agreement suggested that all raters contributed to the variation in calculated risks.

Conclusion: The current results indicate that the preoperative risk calculators POSPOM, SRC, and SORT exhibit poor to moderate interrater reliability. These calculators are not sufficiently accurate for clinical identification and preoperative counseling of high-risk surgical patients. Clinicians should be trained in using mortality risk calculators. Also, clinicians should be cautious when using predicted mortality estimates from these calculators to identify high-risk noncardiac surgical patients for elective surgery.

对用于择期高风险非心脏手术患者的术前死亡率风险计算器进行的术者间可靠性分析表明,计算器的可靠性从较差到中等不等。
背景:网上有多种术前计算器可预测非心脏手术患者的术前死亡风险。然而,目前还不清楚这些风险计算器在不同评分者之间的表现如何。本研究调查了三种术前死亡风险计算器在择期高风险非心脏手术患者群体中的评分者间可靠性,以评估这些计算器能否安全地用于识别高风险非心脏手术患者,以便术前多学科团队讨论:五位麻醉师使用术前计算术后死亡风险评分法(POSPOM)、美国外科学院手术风险计算器(SRC)和手术结果风险工具(SORT)评估了 34 位高风险患者的术前死亡风险。每个计算器共收集了 170 个计算结果:结果:SORT(ICC (C.I. 95%) = 0.46 (0.30-0.63))的交互可靠性较差,SRC(ICC = 0.65 (0.51-0.78))和 POSPOM(ICC = 0.63 (0.49-0.77))的交互可靠性中等。计算出的死亡风险绝对值范围为:POSPOM 0.2-72%,SRC 0-36%,SORT 0.4-17%。POSPOM 和 SORT 的变异系数在风险等级越高时越大。扩展的布兰-阿尔特曼(Bland-Altman)一致性界限表明,所有评分者都对计算风险的差异做出了贡献:目前的研究结果表明,术前风险计算器 POSPOM、SRC 和 SORT 的评分者间可靠性为中差。这些计算器在临床识别和术前咨询高风险手术患者方面不够准确。临床医生应接受使用死亡率风险计算器的培训。此外,临床医生在使用这些计算器预测的死亡率估算值来确定接受择期手术的高风险非心脏外科患者时应谨慎。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
BMC Anesthesiology
BMC Anesthesiology ANESTHESIOLOGY-
CiteScore
3.50
自引率
4.50%
发文量
349
审稿时长
>12 weeks
期刊介绍: BMC Anesthesiology is an open access, peer-reviewed journal that considers articles on all aspects of anesthesiology, critical care, perioperative care and pain management, including clinical and experimental research into anesthetic mechanisms, administration and efficacy, technology and monitoring, and associated economic issues.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信