From Promising Capabilities to Pervasive Bias: Assessing Large Language Models for Emergency Department Triage.

Abstract

Large Language Models (LLMs) have shown significant promise for clinical applications, yet their application to triage remains underexplored. In this study, we systematically investigate the capabilities of LLMs in emergency department triage through two key dimensions: (1) robustness to distribution shifts and missing data, and (2) intersectional biases across sex and race. We assess multiple LLM-based approaches, ranging from continued pre-training to in-context learning, as well as conventional machine learning (ML) approaches. First, we demonstrate that LLMs exhibit superior robustness compared to traditional ML, which is promising due to their ability to provide explanatory rationales. Second, we show that the most effective LLM-based methods are those that select similar examples from prior patient cases, whereas reasoning capabilities in LLMs offer little benefit for triage. Lastly, we identify critical gaps in LLM preferences that emerge at the intersections of sex and race. LLMs exhibit sex-based differences, and they are more pronounced in certain racial groups, suggesting that LLMs encode preferences that emerge in specific clinical contexts and combinations of characteristics. We perform this audit through counterfactual analysis, providing a systematic way to identify such biases before real-world integration.