Consulting the Digital Doctor: Efficacy of ChatGPT-3.5 in Answering Questions Related to Diabetic Foot Ulcer Care.

Abstract

Background: Diabetic foot ulcer (DFU) care is a challenge in reconstructive surgery. Artificial intelligence (AI) tools represent a new resource for patients with DFUs to seek information.

Objective: To evaluate the efficacy of ChatGPT-3.5 in responding to frequently asked questions related to DFU care.

Methods: Researchers posed 11 DFU care questions to ChatGPT-3.5 in December 2023. Questions were divided into topic categories of wound care, concerning symptoms, and surgical management. Four plastic surgeons in the authors' wound care department evaluated responses on a 10-point Likert-type scale for accuracy, comprehensiveness, and danger, in addition to providing qualitative feedback. Readability was assessed using 10 readability indexes.

Results: ChatGPT-3.5 answered questions with a mean accuracy of 8.7±0.3, comprehensiveness of 8.0±0.7, and danger of 2.2±0.6. ChatGPT-3.5 answered at the mean grade level of 11.9±1.8. Physician reviewers complimented the simplicity of the responses (n=11/11) and the AI's ability to provide general information (n=4/11). Three responses presented incorrect information, and the majority of responses (n=10/11) left out key information, such as deep vein thrombosis symptoms and comorbid conditions impacting limb salvage.

Conclusions: The researchers observed that ChatGPT-3.5 provided misinformation, omitted crucial details, and responded at nearly 4 grade levels higher than the American average. However, ChatGPT-3.5 was sufficient in its ability to provide general information, which may enable patients with DFUs to make more informed decisions and better engage in their care. Physicians must proactively address the potential benefits and limitations of AI.