Quality of Information in Carpal Tunnel Syndrome: Social Media Platforms Versus Large Language Models.

Abstract

Introduction: Carpal tunnel syndrome (CTS) is the most common peripheral nerve entrapment disease, and it is a subject of great interest and concern to medical professionals and the general public. Our study aims to analyze and compare the quality and accuracy of the information related to CTS provided by social media platforms (SMPs) and the new large language models (LLM).

Methods: On YouTube, the first 20 videos in English and the first 20 videos in Spanish when searching for "carpal tunnel syndrome" and "síndrome túnel carpo" were selected. On Instagram, the first 20 videos with the hashtag #carpaltunnelsyndrome and #tunelcarpiano were chosen (in total 80 videos). Duration, number of likes, number of views, number of followers, upload date, and author category (medical specialist, patient, etc) were evaluated. Three specific questions about CTS were asked to 2 new LLMs (ChatGPT and Google Bard). The quality of information was analyzed and compared by two independent board-certified plastic surgeons using the Journal of American Medical Association (JAMA) and DISCERN scales.

Results: LLMs showed a significant higher quality of information when compared with SMPs based on the DISCERN scores ( P < 0.05). Average DISCERN scores for answers given by ChatGPT and Google Bard were 52.83 and 57.83, respectively (good quality). In YouTube and Instagram, the average score for the 80 videos based on the JAMA scale was 1.92 (low reliability) and 25.18 (very low quality) on the DISCERN scale. Videos created by medical professionals in SMPs were associated with a higher JAMA and DISCERN scores ( P < 0.05). 53.8% of the videos were made by a nonmedical author.

Conclusions: The quality of information from LLMs was good and significantly better than in SMP. A low participation of board-certified surgeons in SMP was found. Board-certified surgeons should be more involved in LLM and SMPs to increase leadership, improve education, and spread knowledge of peripheral nerve surgery.