Assessing the quality and readability of patient information available for shockwave lithotripsy: Human versus artificial intelligence comparison of resources and quality measurements - A benchmarking study from European association of urology (EAU) endourology.

Abstract

Shockwave lithotripsy (SWL) is a noninvasive method for removing stones, primarily used for small, uncomplicated urinary stones, as the number of kidney stone cases continues to rise. With rising patient numbers, online information must be accurate and easy to understand. We assessed the quality and readability of SWL information available online and compared it with leaflets created by generative artificial intelligence (AI). The top 20 search results for four SWL-related terms on Google, Yahoo, and Bing were collected and duplicates removed. Generative Artificial Intelligence (GAI) platforms such as ChatGPT, Gemini, and DeepSeek generated a patient leaflet. Two authors evaluated quality using DISCERN and JAMA scores and readability using Flesch Reading Ease Score (FRES) and Flesch-Kincaid Grade Level (FKGL). Later, ChatGPT reviewed the AI-generated information with DISCERN. A total of 68 websites were included. The quality assessments showed average websites with AI-leaflet DISCERN scores of 52.4 (±8.91) and 58.7 (±5.03), JAMA Benchmark scores of 2.22 (±1.10) and 0, FRES of 52.1 (±13.8) and 51.5 (±8.05), and FKGL scores of 9.04 (±2.36) and 8.93 (±1.10). No significant differences were seen in DISCERN (P = 0.224), FRES (P = 0.652), and FKGL (P = 0.866), but differences in JAMA scores were significant (P = 0.0010). A difference was also noted in the author-rated and the AI-rated DISCERN scores. There is a deficiency in high-quality SWL information that meets global readability standards. As more individuals use GAI-resources trained on online data, it is crucial to improve their quality for patients and carers. Doing so enables them to make informed decisions and promotes their well-being during treatment, resulting in improved comfort and outcomes.