Evaluating an AI-assisted triage workflow for retinal diseases.

Abstract

The clinical value of artificial intelligence (AI) in fundus photography depends on workflow efficiency as well as diagnostic performance. We evaluated an AI-assisted negative-screening workflow for diabetic retinopathy (DR), retinal vein occlusion (RVO), and age-related macular degeneration (AMD) using 6,904 color fundus photographs independently graded by three retina specialists. In this workflow, AI-negative images were classified as negative without ophthalmologist review, whereas AI-positive images were referred for human interpretation. For each reader-disease pair, the reference standard was agreement between the other two readers; discordant cases were excluded (DR, 1.2%-2.3%; RVO, 0.3%-0.6%; AMD, 6.1%-11.6%). The workflow reduced direct ophthalmologist review to 7.3%-8.0% of images for DR, 11.7%-11.9% for RVO, and 13.5%-16.8% for AMD. For DR, sensitivity decreased significantly (0.9168 to 0.8838; difference, -0.0330; 95% confidence interval [CI], -0.0495 to -0.0165; p < 0.001), whereas specificity did not change significantly (0.9961 to 0.9983; p = 0.183). For RVO, sensitivity was unchanged (0.9807) and specificity did not change significantly (0.9985 to 0.9988; p = 0.320). For AMD, sensitivity decreased significantly (0.8570 to 0.8445; difference, -0.0124; 95% CI, -0.0215 to -0.0033; p = 0.008), whereas specificity did not change significantly (0.9685 to 0.9843; p = 0.324). In an exploratory image-level analysis for at least one target disease, review decreased to 27.7%-30.1%; sensitivity changed from 0.9122 to 0.9039 (difference, -0.0083; 95% CI, -0.0131 to -0.0035; p < 0.001) and specificity from 0.9659 to 0.9804 (p = 0.334). AI-assisted negative-screening can reduce ophthalmologist workload across multiple retinal diseases, but with disease-specific safety trade-offs: because AI-negative images are not reviewed, reduced sensitivity for DR and AMD means a small proportion of true-positive cases would go undetected, potentially delaying diagnosis and treatment for sight-threatening disease. These results support disease-specific implementation strategies that balance workload reduction against missed positive cases.