Development and validation of a novel tool for identification and categorization of non-technical errors associated with surgical mortality.

IF 8.6 1区医学 Q1 SURGERY

British Journal of Surgery Pub Date : 2024-10-01 DOI:10.1093/bjs/znae253

Jesse D Ey,Victoria Kollias,Matheesha B Herath,Octavia Lee,Martin H Bruening,Adam J Wells,Guy J Maddern

{"title":"Development and validation of a novel tool for identification and categorization of non-technical errors associated with surgical mortality.","authors":"Jesse D Ey,Victoria Kollias,Matheesha B Herath,Octavia Lee,Martin H Bruening,Adam J Wells,Guy J Maddern","doi":"10.1093/bjs/znae253","DOIUrl":null,"url":null,"abstract":"BACKGROUND\r\nUp to half of all surgical adverse events are due to non-technical errors, making non-technical skill assessment and improvement a priority. No specific tools are available to retrospectively identify non-technical errors that have occurred in surgical patient care. This original study aimed to develop and provide evidence of validity and inter-rater reliability for the System for Identification and Categorization of Non-technical Error in Surgical Settings (SICNESS).\r\n\r\nMETHODS\r\nA literature review, modified Delphi process, and two pilot phases were used to develop and test the SICNESS tool. For each pilot, 12 months of surgical mortality data from the Australian and New Zealand Audit of Surgical Mortality were assessed by two independent reviewers using the SICNESS tool. Main outcomes included tool validation through modified Delphi consensus, and inter-rater reliability for: non-technical error identification and non-technical error categorization using Cohen's κ coefficient, and overall agreement using Fleiss' κ coefficient.\r\n\r\nRESULTS\r\nVersion 1 of the SICNESS was used for pilot 1, including 412 mortality cases, and identified and categorized non-technical errors with strong-moderate inter-rater reliability. Non-technical error exemplars were created and validated through Delphi consensus, and a novel mental model was developed. Pilot 2 included an additional 432 mortality cases. Inter-rater reliability was near perfect for leadership (κ 0.92, 95% c.i. 0.82 to 1.00); strong for non-technical error identification (κ 0.89, 0.84 to 0.93), communication and teamwork (κ 0.89, 0.79 to 0.99), and decision-making (κ 0.85, 0.79 to 0.92); and moderate for situational awareness (κ 0.79, 0.71 to 0.87) and overall agreement (κ 0.69, 0.66 to 0.73).\r\n\r\nCONCLUSION\r\nThe SICNESS is a reliable and valid tool, enabling retrospective identification and categorization of non-technical errors associated with death, occurring in real surgical patient interactions.","PeriodicalId":136,"journal":{"name":"British Journal of Surgery","volume":"233 1","pages":""},"PeriodicalIF":8.6000,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"British Journal of Surgery","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1093/bjs/znae253","RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"SURGERY","Score":null,"Total":0}

引用次数: 0

Abstract

BACKGROUND Up to half of all surgical adverse events are due to non-technical errors, making non-technical skill assessment and improvement a priority. No specific tools are available to retrospectively identify non-technical errors that have occurred in surgical patient care. This original study aimed to develop and provide evidence of validity and inter-rater reliability for the System for Identification and Categorization of Non-technical Error in Surgical Settings (SICNESS). METHODS A literature review, modified Delphi process, and two pilot phases were used to develop and test the SICNESS tool. For each pilot, 12 months of surgical mortality data from the Australian and New Zealand Audit of Surgical Mortality were assessed by two independent reviewers using the SICNESS tool. Main outcomes included tool validation through modified Delphi consensus, and inter-rater reliability for: non-technical error identification and non-technical error categorization using Cohen's κ coefficient, and overall agreement using Fleiss' κ coefficient. RESULTS Version 1 of the SICNESS was used for pilot 1, including 412 mortality cases, and identified and categorized non-technical errors with strong-moderate inter-rater reliability. Non-technical error exemplars were created and validated through Delphi consensus, and a novel mental model was developed. Pilot 2 included an additional 432 mortality cases. Inter-rater reliability was near perfect for leadership (κ 0.92, 95% c.i. 0.82 to 1.00); strong for non-technical error identification (κ 0.89, 0.84 to 0.93), communication and teamwork (κ 0.89, 0.79 to 0.99), and decision-making (κ 0.85, 0.79 to 0.92); and moderate for situational awareness (κ 0.79, 0.71 to 0.87) and overall agreement (κ 0.69, 0.66 to 0.73). CONCLUSION The SICNESS is a reliable and valid tool, enabling retrospective identification and categorization of non-technical errors associated with death, occurring in real surgical patient interactions.

查看原文本刊更多论文

开发并验证一种新型工具，用于识别与手术死亡率相关的非技术性错误并对其进行分类。

背景多达一半的手术不良事件是由非技术性错误造成的，因此非技术性技能的评估和改进成为当务之急。目前还没有专门的工具来回顾性地识别外科患者护理中发生的非技术性错误。本原创性研究旨在开发外科非技术性错误识别和分类系统（SICNESS），并为其有效性和评分者间可靠性提供证据。方法采用文献综述、修改后的德尔菲流程和两个试点阶段来开发和测试 SICNESS 工具。在每个试点阶段，由两名独立评审员使用 SICNESS 工具对澳大利亚和新西兰手术死亡率审计中 12 个月的手术死亡率数据进行评估。主要结果包括通过修改后的德尔菲共识对工具进行验证，以及使用 Cohen's κ coefficient 进行非技术性错误识别和非技术性错误分类的评分者间可靠性，以及使用 Fleiss's κ coefficient 进行总体一致性。通过德尔菲共识创建和验证了非技术性错误示例，并开发了一个新的心理模型。试验 2 包括另外 432 个死亡病例。在领导能力（κ 0.92，95% c.i.0.82-1.00）、非技术性错误识别（κ 0.89，0.84-0.93）、沟通和团队合作（κ 0.89，0.79-0.99）以及决策（κ 0.85，0.79-0.92）方面，评分者之间的可靠性接近完美；在情景意识（κ 0.结论 SICNESS 是一种可靠有效的工具，能够对实际手术患者互动中发生的与死亡相关的非技术性错误进行回顾性识别和分类。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

British Journal of Surgery 医学-外科

CiteScore

12.70

自引率

7.30%

发文量

1102

审稿时长

1.5 months

期刊介绍： The British Journal of Surgery (BJS), incorporating the European Journal of Surgery, stands as Europe's leading peer-reviewed surgical journal. It serves as an invaluable platform for presenting high-quality clinical and laboratory-based research across a wide range of surgical topics. In addition to providing a comprehensive coverage of traditional surgical practices, BJS also showcases emerging areas in the field, such as minimally invasive therapy and interventional radiology. While the journal appeals to general surgeons, it also holds relevance for specialty surgeons and professionals working in closely related fields. By presenting cutting-edge research and advancements, BJS aims to revolutionize the way surgical knowledge is shared and contribute to the ongoing progress of the surgical community.