ChatGPT-4o and 4o1 Preview as Dietary Support Tools in a Real-World Medicated Obesity Program: A Prospective Comparative Analysis.

Abstract

Background/objectives: Clinicians are becoming increasingly interested in the use of large language models (LLMs) in obesity services. While most experts agree that LLM integration would increase access to obesity care and its efficiency, many remain skeptical of their scientific accuracy and capacity to convey human empathy. Recent studies have shown that ChatGPT-3 models are capable of emulating human dietitian responses to a range of basic dietary questions.

Methods: This study compared responses of two ChatGPT-4o models to those from human dietitians across 10 complex questions (5 broad; 5 narrow) derived from patient-clinician interactions within a real-world medicated digital weight loss service.

Results: Investigators found that neither ChatGPT-4o nor Chat GPT-4o1 preview were statistically outperformed (p < 0.05) by human dietitians on any of the study's 10 questions. The same finding was made when scores were aggregated from the ten questions across the following four individual study criteria: scientific correctness, comprehensibility, empathy/relatability, and actionability.

Conclusions: These results provide preliminary evidence that advanced LLMs may be able to play a significant supporting role in medicated obesity services. Research in other obesity contexts is needed before any stronger conclusions are made about LLM lifestyle coaching and whether such initiatives increase care access.