Reliability and quality of information provided by artificial intelligence chatbots on post-contrast acute kidney injury: an evaluation of diagnostic, preventive, and treatment guidance.

Abstract

Objective: The aim of this study was to evaluate the reliability and quality of information provided by artificial intelligence chatbots regarding the diagnosis, preventive methods, and treatment of contrast-associated acute kidney injury, while also discussing their benefits and drawbacks.

Methods: The most frequently asked questions regarding contrast-associated acute kidney injury on Google Trends between January 2022 and January 2024 were posed to four artificial intelligence chatbots: ChatGPT, Gemini, Copilot, and Perplexity. The responses were evaluated based on the DISCERN score, the Patient Education Materials Assessment Tool for Printable Materials score, the Web Resource Rating scale, the Coleman-Liau index, and a Likert scale.

Results: As per the DISCERN score, the quality of information provided by Perplexity received a rating of "good", while the quality of information acquired by ChatGPT, Gemini, and Copilot was scored as "average." Based on the Coleman-Liau index, the readability of the responses was greater than 11 for all artificial intelligence chatbots, suggesting a high level of complexity requiring a university-level education. Similarly, the understandability and applicability scores on the Patient Education Materials Assessment Tool for Printable Materials and the Web Resource Rating scale were low for all artificial intelligence programs. In consideration of the Likert score, all artificial intelligence chatbots received favorable ratings.

Conclusions: While patients increasingly utilize artificial intelligence chatbots to acquire information about contrast-associated acute kidney injury, the readability and understandability of the information provided may be low.