Kelly M Toth, Zahra Aghababa, Jason N Kennedy, Chukwudi Onyemekwu, Niall T Prendergast, Christopher A Franz, Michael E Reznik, Brian Jiang, Brett Curtis, Faraaz Shah, Georgios D Kitsios, Bryan J McVerry, Timothy D Girard
{"title":"优化床边护士记录和训练有素的研究者在ICU谵妄评估之间的一致性。","authors":"Kelly M Toth, Zahra Aghababa, Jason N Kennedy, Chukwudi Onyemekwu, Niall T Prendergast, Christopher A Franz, Michael E Reznik, Brian Jiang, Brett Curtis, Faraaz Shah, Georgios D Kitsios, Bryan J McVerry, Timothy D Girard","doi":"10.1097/CCM.0000000000006879","DOIUrl":null,"url":null,"abstract":"<p><strong>Objectives: </strong>Delirium is common and harmful in the ICU. The Intensive Care Delirium Screening Checklist (ICDSC) and Confusion Assessment Method for the ICU (CAM-ICU) are validated tools recommended for delirium identification. However, the accuracy of bedside nurse-documented delirium assessments in the ICU is inconsistent, limiting utility in clinical research. We sought to evaluate and optimize agreement between bedside nurse-documented and trained researcher delirium assessments.</p><p><strong>Design, setting, and patients: </strong>Critically ill adults with acute respiratory failure or sepsis in ICUs in large academic hospitals in a southwestern Pennsylvania health system were assessed daily for delirium by bedside nurses (using the ICDSC) and trained researchers (using the CAM-ICU). Using matched nurse-to-researcher delirium assessments, we categorized delirium status using validated cutoffs and evaluated agreement using Cohen's kappa. We derived and compared logistic regression models that used ICDSC documentation, mechanical ventilation status, and admission Sequential Organ Failure Assessment to predict delirium in noncomatose patients, using researcher CAM-ICU assessments as the reference standard. We internally validated models using ten-fold cross-validation.</p><p><strong>Interventions: </strong>None.</p><p><strong>Measurements and main results: </strong>From a sample of 1535 matched assessments of 279 patients, there was moderate agreement between bedside nurse assessments using the established ICDSC delirium/normal cutoff (ICDSC ≥ 4) and trained researcher assessments using the CAM-ICU (Cohen's kappa = 0.42). A logistic regression model informed by individual ICDSC components and clinical data predicted a positive research CAM-ICU with good discrimination (area under the curve = 0.87) and performed well in cross-validation (F1 score = 0.72). In sensitivity analyses, models with more limited ICDSC information demonstrated fair to good discriminatory ability (F1 = 0.60-0.70), with the validated cutoff model having the lowest performance.</p><p><strong>Conclusions: </strong>A delirium model informed by bedside nurse ICDSC findings and clinical variables improves accuracy of delirium detected in the ICU and can be used in future pragmatic research that leverages large clinical datasets to advance understanding of delirium mechanisms, trajectories, and outcomes.</p>","PeriodicalId":10765,"journal":{"name":"Critical Care Medicine","volume":" ","pages":""},"PeriodicalIF":6.0000,"publicationDate":"2025-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Optimizing Agreement Between Bedside Nurse-Documented and Trained Researcher Delirium Assessments in the ICU.\",\"authors\":\"Kelly M Toth, Zahra Aghababa, Jason N Kennedy, Chukwudi Onyemekwu, Niall T Prendergast, Christopher A Franz, Michael E Reznik, Brian Jiang, Brett Curtis, Faraaz Shah, Georgios D Kitsios, Bryan J McVerry, Timothy D Girard\",\"doi\":\"10.1097/CCM.0000000000006879\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>Objectives: </strong>Delirium is common and harmful in the ICU. The Intensive Care Delirium Screening Checklist (ICDSC) and Confusion Assessment Method for the ICU (CAM-ICU) are validated tools recommended for delirium identification. However, the accuracy of bedside nurse-documented delirium assessments in the ICU is inconsistent, limiting utility in clinical research. We sought to evaluate and optimize agreement between bedside nurse-documented and trained researcher delirium assessments.</p><p><strong>Design, setting, and patients: </strong>Critically ill adults with acute respiratory failure or sepsis in ICUs in large academic hospitals in a southwestern Pennsylvania health system were assessed daily for delirium by bedside nurses (using the ICDSC) and trained researchers (using the CAM-ICU). Using matched nurse-to-researcher delirium assessments, we categorized delirium status using validated cutoffs and evaluated agreement using Cohen's kappa. We derived and compared logistic regression models that used ICDSC documentation, mechanical ventilation status, and admission Sequential Organ Failure Assessment to predict delirium in noncomatose patients, using researcher CAM-ICU assessments as the reference standard. We internally validated models using ten-fold cross-validation.</p><p><strong>Interventions: </strong>None.</p><p><strong>Measurements and main results: </strong>From a sample of 1535 matched assessments of 279 patients, there was moderate agreement between bedside nurse assessments using the established ICDSC delirium/normal cutoff (ICDSC ≥ 4) and trained researcher assessments using the CAM-ICU (Cohen's kappa = 0.42). A logistic regression model informed by individual ICDSC components and clinical data predicted a positive research CAM-ICU with good discrimination (area under the curve = 0.87) and performed well in cross-validation (F1 score = 0.72). In sensitivity analyses, models with more limited ICDSC information demonstrated fair to good discriminatory ability (F1 = 0.60-0.70), with the validated cutoff model having the lowest performance.</p><p><strong>Conclusions: </strong>A delirium model informed by bedside nurse ICDSC findings and clinical variables improves accuracy of delirium detected in the ICU and can be used in future pragmatic research that leverages large clinical datasets to advance understanding of delirium mechanisms, trajectories, and outcomes.</p>\",\"PeriodicalId\":10765,\"journal\":{\"name\":\"Critical Care Medicine\",\"volume\":\" \",\"pages\":\"\"},\"PeriodicalIF\":6.0000,\"publicationDate\":\"2025-09-19\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Critical Care Medicine\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1097/CCM.0000000000006879\",\"RegionNum\":1,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"CRITICAL CARE MEDICINE\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Critical Care Medicine","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1097/CCM.0000000000006879","RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"CRITICAL CARE MEDICINE","Score":null,"Total":0}
Optimizing Agreement Between Bedside Nurse-Documented and Trained Researcher Delirium Assessments in the ICU.
Objectives: Delirium is common and harmful in the ICU. The Intensive Care Delirium Screening Checklist (ICDSC) and Confusion Assessment Method for the ICU (CAM-ICU) are validated tools recommended for delirium identification. However, the accuracy of bedside nurse-documented delirium assessments in the ICU is inconsistent, limiting utility in clinical research. We sought to evaluate and optimize agreement between bedside nurse-documented and trained researcher delirium assessments.
Design, setting, and patients: Critically ill adults with acute respiratory failure or sepsis in ICUs in large academic hospitals in a southwestern Pennsylvania health system were assessed daily for delirium by bedside nurses (using the ICDSC) and trained researchers (using the CAM-ICU). Using matched nurse-to-researcher delirium assessments, we categorized delirium status using validated cutoffs and evaluated agreement using Cohen's kappa. We derived and compared logistic regression models that used ICDSC documentation, mechanical ventilation status, and admission Sequential Organ Failure Assessment to predict delirium in noncomatose patients, using researcher CAM-ICU assessments as the reference standard. We internally validated models using ten-fold cross-validation.
Interventions: None.
Measurements and main results: From a sample of 1535 matched assessments of 279 patients, there was moderate agreement between bedside nurse assessments using the established ICDSC delirium/normal cutoff (ICDSC ≥ 4) and trained researcher assessments using the CAM-ICU (Cohen's kappa = 0.42). A logistic regression model informed by individual ICDSC components and clinical data predicted a positive research CAM-ICU with good discrimination (area under the curve = 0.87) and performed well in cross-validation (F1 score = 0.72). In sensitivity analyses, models with more limited ICDSC information demonstrated fair to good discriminatory ability (F1 = 0.60-0.70), with the validated cutoff model having the lowest performance.
Conclusions: A delirium model informed by bedside nurse ICDSC findings and clinical variables improves accuracy of delirium detected in the ICU and can be used in future pragmatic research that leverages large clinical datasets to advance understanding of delirium mechanisms, trajectories, and outcomes.
期刊介绍:
Critical Care Medicine is the premier peer-reviewed, scientific publication in critical care medicine. Directed to those specialists who treat patients in the ICU and CCU, including chest physicians, surgeons, pediatricians, pharmacists/pharmacologists, anesthesiologists, critical care nurses, and other healthcare professionals, Critical Care Medicine covers all aspects of acute and emergency care for the critically ill or injured patient.
Each issue presents critical care practitioners with clinical breakthroughs that lead to better patient care, the latest news on promising research, and advances in equipment and techniques.