Leveraging social media and large language models for scalable alcohol risk assessment: Examining validity with AUDIT-C and post recency effects

Abstract

Risky alcohol consumption is a major public health concern, yet significant barriers exist to effective screening. The present study examines the potential of Large Language Models (LLMs) to infer risky alcohol use from social media text. The unobtrusive nature of this approach could provide a more scalable way to assess alcohol risk in large populations. To this aim, we analyzed Facebook status updates from 208 adults from Italy (mean age = 26.8, 70.7 % female) who also completed the Alcohol Use Disorders Identification Test-Consumption (AUDIT-C), a brief validated self-report measure of risky drinking. Two state-of-the-art LLMs, Gemini 1.5 Pro and GPT-4o, were used to assess alcohol risk and to quantify alcohol references. Results demonstrated strong inter-model agreement between risk inferences (ρ = 0.572, p < 0.001). LLM-inferred risk scores showed moderate correlations with AUDIT-C scores (Gemini 1.5 Pro: ρ = 0.344, p < 0.001; GPT-4o: ρ = 0.375, p < 0.001; Average: ρ = 0.405, p < 0.001). These correlations were significantly stronger among participants with recent posts (Average risk score: ρ = 0.500, p < 0.001) than among those without (ρ = 0.294, p = 0.008). The strongest correlation was observed between average LLM-inferred risk scores and AUDIT-C in the recent posts group (disattenuated ρ = 0.606). These findings suggest that LLMs offer a promising tool for identifying risky alcohol use when analyzing recent social media activity. Their accuracy is comparable to some traditional alcohol assessment methods, highlighting their potential to enhance early detection efforts. Limitations and future research directions are discussed.