Evaluating Text-to-Image Generation in Pediatric Ophthalmology.

Abstract

Purpose: To evaluate the quality and accuracy of artificial intelligence (AI)-generated images depicting pediatric ophthalmology pathologies compared to human-illustrated images, and assess the readability, quality, and accuracy of accompanying AI-generated textual information.

Methods: This cross-sectional comparative study analyzed outputs from DALL·E 3 (OpenAI) and Gemini Advanced (Google). Nine pediatric ophthalmology pathologies were sourced from the American Association for Pediatric Ophthalmology and Strabismus (AAPOS) "Most Common Searches." Two prompts were used: Prompt A asked large language models (LLMs), "What is [insert pathology]?" Prompt B requested text-to-image generators (TTIs) to create images of the pathologies. Textual responses were evaluated for quality using published criteria (helpfulness, truthfulness, harmlessness; score 1 to 15, ≥ 12: high quality) and readability using Simple Measure of Gobbledygook (SMOG) and Flesch-Kincaid Grade Level (≤ 6th-grade level: readable). Images were assessed for anatomical accuracy, pathological accuracy, artifacts, and color (score 1 to 15, ≥ 12: high quality). Human-illustrated images served as controls.

Results: DALL·E 3 images were of poor quality (median: 7; range: 3 to 15) and significantly worse than human-illustrated controls (median: 15; range: 9 to 15; P < .001). Pathological accuracy was also poor (median: 1). Textual information from ChatGPT-4o and Gemini Advanced was high quality (median: 15) but difficult to read (Chat-GPT-4o: SMOG: 8.2, FKGL: 8.9; Gemini Advanced: SMOG: 8.5, FKGL: 9.3).

Conclusions: Text-to-image generators are poor at generating images of common pediatric ophthalmology pathologies. They can serve as adequate supplemental tools for generating high-quality accurate textual information, but care must be taken to tailor generated text to be readable by users.