Performance of a novel medical artificial intelligence large language model on supporting decision-making for emergency patients with suspected sepsis.

Abstract

Background: Large language models (LLMs) are being explored for disease prediction and diagnosis; however, their efficacy for early sepsis identification in emergency departments (EDs) remains unexplored. This study aims to evaluate MedGo, a novel medical LLM, as a decision-support tool for clinicians managing patients with suspected sepsis.

Methods: This retrospective study included anonymized medical records of 203 patients (mean age 79.9±10.2 years) with confirmed sepsis from a tertiary hospital ED between January 2023 and January 2024. MedGo performance across nine sepsis-related assessment tasks was compared with that of two junior (<3 years of experience) and two senior (>10 years of experience) ED physicians. Assessments were scored on a 5-point Likert scale for accuracy, comprehensiveness, readability, and case-analysis skills.

Results: MedGo demonstrated diagnostic performance comparable to that of senior physicians across most metrics, achieving a median Likert score of 4 in accuracy, comprehensiveness, and readability. MedGo significantly outperformed junior physicians (P<0.001 for accuracy and case-analysis skills). MedGo assistance significantly enhanced both junior (P<0.001) and senior (P<0.05) physicians' diagnostic accuracy. Notably, MedGo-assisted junior physicians achieved accuracy levels comparable to those of unassisted senior physicians. MedGo maintained consistent performance across varying sepsis severities.

Conclusion: MedGo shows significant diagnostic efficacy for sepsis and effectively supports clinicians in the ED, particularly enhancing junior physicians' performance. Our study highlights the potential of MedGo as a valuable decision-support tool for sepsis management, paving the way for specialized sepsis AI models.