Integrating clinical guidelines with large language models for improved sepsis mortality prediction.

Abstract

We develop and validate a clinical guideline-integrated LLM for enhanced sepsis mortality prediction. Using MIMIC-IV data from 24,237 ICU sepsis patients, we fine-tuned a large language model with Low-Rank Adaptation, embedding clinical guidelines into the training process. The model's predictive performance was evaluated using accuracy, F1-score, sensitivity, specificity, and area under the receiver operating characteristic curve (AUC). Ablation studies assessed the specific contributions of clinical guideline integration. The guideline-enhanced fine-tuned LLM demonstrated moderately higher performance across all evaluation metrics including predictive accuracy (0.819), F1-score (0.815), sensitivity (0.815), specificity (0.822), and AUC (0.852) in predicting mortality risk for septic patients compared to traditional machine learning (highest accuracy: 0.774, AUC: 0.850) and deep learning methods (highest accuracy: 0.762, AUC: 0.841). Ablation experiments demonstrated that explicit integration of clinical guideline knowledge substantially improved performance over both direct prompting (accuracy: 0.709, AUC: 0.706) and fine-tuning without clinical guidelines (accuracy: 0.786, AUC: 0.801). These findings demonstrate that incorporating clinical guidelines into the fine-tuning of large language models outperforms both traditional and deep learning baselines across multiple metrics in sepsis mortality prediction, highlighting the value of explicit domain knowledge integration for clinical AI's robustness.