Machine learning outperforms the Canadian Triage and Acuity Scale (CTAS) in predicting need for early critical care.

Abstract

Study objective: This study investigates the potential to improve emergency department (ED) triage using machine learning models by comparing their predictive performance with the Canadian Triage Acuity Scale (CTAS) in identifying the need for critical care within 12 h of ED arrival.

Methods: Three machine learning models (LASSO regression, gradient-boosted trees, and a deep learning model with embeddings) were developed using retrospective data from 670,841 ED visits to the Jewish General Hospital from June 2012 to Jan 2021. The model outcome was the need for critical care within the first 12 h of ED arrival. Metrics, including the areas under the receiver-operator characteristic curve (ROC) and precision-recall curve (PRC) were used for performance evaluation. Shapley additive explanation scores were used to compare predictor importance.

Results: The three machine learning models (deep learning, gradient-boosted trees and LASSO regression) had areas under the ROC of 0.926 ± 0.003, 0.912 ± 0.003 and 0.892 ± 0.004 respectively, and areas under the PRC of 0.27 ± 0.01, 0.24 ± 0.01 and 0.23 ± 0.01 respectively. In comparison, the CTAS score had an area under the ROC of 0.804 ± 0.006 and under the PRC of 0.11 ± 0.01. The predictors of most importance were similar between the models.

Conclusions: Machine learning models outperformed CTAS in identifying, at the point of ED triage, patients likely to need early critical care. If validated in future studies, machine learning models such as the ones developed here may be considered for incorporation in future revisions of the CTAS triage algorithm, potentially improving discrimination and reliability.