Large language models approach specialist-level accuracy in dental pain diagnosis: a comparative evaluation of four models.

Abstract

Background: Dental pain is the most common chief complaint in oral healthcare and involves multiple disease types. An accurate diagnosis is crucial for treatment decisions. Large language models (LLMs) have great potential for medical diagnosis; however, their accuracy in dental pain diagnosis has not been systematically evaluated. The objectives of this study were to evaluate the diagnostic accuracy of four advanced LLMs for dental pain and analyze the key factors affecting diagnostic performance.

Methods: A comparative study was conducted using 320 dental pain cases from authoritative sources covering eight disease categories. Four state-of-the-art LLMs were evaluated: DeepSeek-R1, ChatGPT-4o, Claude-3.5 Sonnet, and Gemini Pro 2.0. A blinded evaluation was performed by senior experts. The primary outcome was overall diagnostic accuracy, and the secondary outcomes included performance across disease types, complexity levels, and confidence levels.

Results: DeepSeek-R1 achieved the highest diagnostic accuracy (83.8%), followed by ChatGPT-4o (82.8%), with no significant differences. Pulpal disease had the highest accuracy (88.1%), whereas neurological pain had the lowest (59.4%). The case complexity significantly affects the accuracy. The AI models demonstrated significant time advantages, representing a 94.2% faster diagnostic speed than human experts.

Conclusion: Latest-generation LLMs, particularly DeepSeek-R1 and ChatGPT-4o, achieved diagnostic levels approaching those of specialist physicians while providing substantial time efficiency gains. These findings support the application of AI in oral medicine by using confidence-based stratified implementation strategies.