Pilot study evaluating the usability of MonŒil, a ChatGPT-based education tool in ophthalmology

Abstract

Importance

There is an increasing use of artificial intelligence (AI) in ophthalmology to respond to the needs of patients to access reliable, easy-to-understand medical information.

Objective

To assess patient satisfaction with the usability of MonŒil, an AI-based platform designed for patient education in ophthalmology.

Design

This was a pilot cross-sectional study of the usability of MonŒil by patients followed for advanced age-related macular degeneration (AMD). MonŒil is based on ChatGPT-4 technology with specific ophthalmology-focused customizations and accessibility enhancements, and is freely available at monoeil.help. Patients were given 20 min of unsupervised interaction with no prior training or guidance, after which their feedback was collected.

Setting

The study was performed in the ophthalmology department at the Creteil University Hospital.

Participants

Participants included 54 patients diagnosed with advanced AMD defined presenting for follow-up. Patients had to be older than 50 years of age, capable of giving informed consent, and able to understand and interact with MonŒil. Exclusion criteria were severe visual and cognitive impairment that prevented interaction with MonŒil.

Main Outcome(s) and Measure(s)

The primary outcome was the usability of MonŒil as measured by the System Usability Scale (SUS) questionnaire.

Results

Of the 54 participants, 34 were female (62.96 %). The mean age of the cohort was 77.76±8.14 years (range 58 to 97 years). The mean SUS score was 90.23±12.04 with a median of 92.50 (range 42.50 to 100.00), indicating excellent usability. There was a positive relationship between visual acuity and SUS score (regression coefficient 0.30 (95 % CI 0.08 to 0.51), r2=0.19, p = 0.0077).

Conclusions and Relevance

MonŒil demonstrated excellent usability and satisfaction in a sample population of elderly patients with advanced AMD. These results suggest that AI-based tools like MonŒil can enhance patient education with minimal oversight in a complex field like ophthalmology, supporting its use as an adjunct to the physician-patient discussion. Further studies may be necessary to establish the applicability of MonŒil to a broader user base, and to assess its usefulness and clinical impact on patient outcomes such as patient knowledge and vision-related quality of life.