Assessing ChatGPT's suitability in responding to the public's inquires on the effects of smoking on oral health.

Abstract

Background: With the growing use of Artificial Intelligence (AI) in healthcare, language models like ChatGPT are increasingly being used to provide health information. Given smoking's significant impact on oral health, including periodontal disease, tooth loss, oral cancer, and impaired healing-this study assesses ChatGPT's effectiveness in answering inquiries about the effects of smoking on oral health.

Methods: A total of 119 smoking-related dental health questions were generated using online tools and categorized into five domains: periodontal conditions, teeth and health, oral hygiene and breath, oral soft tissues, and oral surgery. ChatGPT 3.5 responses were evaluated for usefulness, readability (FKG index and, SMOG index), quality (Global Quality Scale), reliability (CLEAR tool), and understandability/actionability (PEMAT). Statistical analyses included ANOVA, Kruskal-Wallis, Mann-Whitney U, and Spearman correlation tests using SPSS v20.

Results: ChatGPT responses were rated as "very useful" (36.1%) or "useful" (42.0%) for most questions. The oral hygiene and breath category received the highest usefulness scores (p =.002). Readability scores indicated that 35.3% of responses were very difficult to read. Surgery-related questions were significantly harder to read (p <.001). The overall quality and reliability of responses was moderate to good. Although understandability was generally high, actionability was limited, with only 23.5% of responses achieving high actionability (> 70%).

Conclusion: ChatGPT provides reliable, high-quality information on smoking's impact on oral health, suitable for general health education. However, it lacks actionable, personalized guidance, limiting its effectiveness in supporting behavioral change and health management.