Choice of Machine Learning Models Is Important to Predict Post-Operative Infections in Surgical Patients.

Abstract

Background: Surgical quality datasets are critical to decision-making tools including surgical infection (SI). Machine learning models (MLMs), a branch of artificial intelligence (AI), are increasingly being ingrained within surgical decision-making algorithms. However, given the unique and distinct functioning of individual models, not all models may be suitable for acutely ill surgical patients. Patients and Methods: This is a 5-year retrospective review of National Surgical Quality Improvement Program (NSQIP) patients who underwent an operation. The data were reviewed for demographics, medical comorbidities, rates, and sites of infection. To generate the MLMs, data were imported into Python, and four common MLMs, extreme gradient boosting, K-nearest neighbor (KNN), random forest, and logistic regression, as well as two novel models (flexible discriminant analysis and generalized additive model) and ensemble modeling, were generated to predict post-operative SIs. Outputs included area under the receiver-operating characteristic curve (AUC ROC) including recall curves. Results: Overall, 624,625 urgent and emergent NSQIP patients were included. The overall infection rate was 8.6%. Patients who sustained a post-operative infection were older, more likely geriatric, male, diabetic, had chronic obstructive pulmonary disease, were smokers, and were less likely White race. With respect to MLMs, all four MLMs had reasonable accuracy. However, a hierarchy of MLMs was noted with predictive abilities (XGB AUC = 0.85 and logistic regression = 0.82), wherein KNN has the lowest performance (AUC = 0.62). With respect to the ability to detect an infection, precision recall of XGB performed well (AUC = 0.73), whereas KNN performed poorly (AUC = 0.16). Conclusions: MLMs are not created nor function similarly. We identified differences with MLMs to predict post-operative infections in surgical patients. Before MLMs are incorporated into surgical decision making, it is critical that surgeons are at the fore of understanding the role and functioning of MLMs.