Assessing the ability of ChatGPT to generate French patient-facing information to improve patient understanding in hand surgery.

Abstract

Introduction: The advent of artificial intelligence technologies, such as ChatGPT and Gemini, presents new opportunities to enhance physician-patient communication through tailored patient-facing medical information (PFI). Effective postoperative care is critical to successful patient outcomes in hand surgery, making it essential to deliver information in a clear, comprehensible, and accurate manner. This study aims to evaluate the ability of ChatGPT to generate accessible PFI to enhance patient understanding in hand surgery.

Methods: A cross-sectional study was conducted involving five French hand surgeons and 28 non-medical individuals. Participants blindly evaluated PFI containing text and images generated by ChatGPT and Gemini compared to those provided by established organizations (EOs). The evaluations focused on three common hand surgeries: carpal tunnel syndrome, Dupuytren's disease, and synovial cyst. Hand surgeons evaluated PFI based on accuracy, clarity, comprehensiveness, and overall preference. Non-medical participants evaluated PFI based on clarity and overall preference. Surveys were used to capture these evaluations, allowing for a systematic and effective comparison between both sources. Readability was analyzed using six readability formulas, and Likert scale responses were statistically analyzed using paired t-tests.

Results: No significant difference was found in terms of accuracy between ChatGPT-generated text responses and EO-provided text responses. However, text responses provided by EOs were rated significantly higher in terms of comprehensiveness and clarity by hand surgeons. Interestingly, non-medical participants rated the clarity and overall preference of ChatGPT-generated text responses higher than those from EOs. EO-provided images were also significantly favored in terms of comprehensiveness and clarity by hand surgeons. Notably, 65% of non-medical participants preferred EO-provided images over AI-generated ones. Both sources, however, produced information that exceeded the recommended readability levels for patient comprehension.

Conclusion: This study underscores the potential of AI-generated medical information to enhance patient understanding, particularly through the comprehensive nature of materials created by tools like ChatGPT. The divergence in preferences between hand surgeons and patients highlights the need to refine the accuracy, clarity, and relevance of AI-generated content to align with the standards upheld by healthcare professionals. Continued exploration in this area is crucial for optimizing patient education and communication, particularly in the context of postoperative care.