Detecting Eye Disease Using Vision Transformers Informed by Ophthalmology Resident Gaze Data.

Abstract

We showcase two proof-of-concept approaches for enhancing the Vision Transformer (ViT) model by integrating ophthalmology resident gaze data into its training. The resulting Fixation-Order-Informed ViT and Ophthalmologist-Gaze-Augmented ViT show greater accuracy and computational efficiency than ViT for detection of the eye disease, glaucoma.Clinical relevance- By enhancing glaucoma detection via our gaze-informed ViTs, we introduce a new paradigm for medical experts to directly interface with medical AI, leading the way for more accurate and interpretable AI 'teammates' in the ophthalmic clinic.