Real-world evaluation of an OCT-based AI decision-support system for neovascular AMD activity triage in teleophthalmology.

Abstract

Purpose: To evaluate real-world agreement between a CE-marked OCT-based AI decision-support system and routine retreatment decisions for neovascular AMD within a teleophthalmology workflow.

Methods: Retrospective clinical study including 429 OCT examinations from 247 patients (306 treated eyes) with neovascular AMD. Retinal specialists made routine retreatment decisions ("inject" vs "watch-and-wait") using full clinical context. Independently, the AI system (deepeye^® TPS, version 1.2) analyzed the current OCT volume only (no prior OCT, visual acuity, treatment interval, or clinical notes) and generated a Disease Activity Score (DAS; 0-100) used to derive an "inject" vs "watch-and-wait" recommendation. Discrepant cases were re-evaluated by senior graders to establish a double-senior-graded (DSG) reference standard. Implementation analyses assessed a deferral ("safety zone") strategy. Main outcome measures included agreement/accuracy, sensitivity, and specificity versus real-world decisions and the double-senior-graded (DSG) reference standard, as well as decision coverage under deferral.

Results: Agreement between real-world decisions and AI recommendations was 83.2% (sensitivity 74.7%, specificity 88.0%). Against the double-senior-graded reference standard (DSG), accuracy in the full analysis set (FAS), analyzed at the examination level, was 85.5% (sensitivity 77.2%, specificity 90.4%). Using an empirically optimized DAS threshold, accuracy increased to 88.6% (sensitivity 78.5%, specificity 94.9%) in the eligible retreatment-decision set (ERDS). Application of a deferral policy ("safety zone", DAS 33-64) resulted in automated recommendations for 78.4% of eligible examinations, while 21.6% were deferred due to intermediate DAS values; among examinations with an automated recommendation, accuracy was 92.3% (sensitivity 83.2%, specificity 97.5%). Most misclassifications involved subtle IRF/SRF and SHRM as identified by the reading center and tended to be underestimated by the AI.

Conclusion: The evaluated OCT-only AI decision-support output showed substantial agreement with routine retinal-specialist retreatment decisions in a real-world teleophthalmology workflow, particularly when intermediate Disease Activity Scores were deferred to human review. However, false-negative cases and context-dependent discrepancies highlight that the system should support, not replace, clinician judgement. Prospective multicenter validation using longitudinal and multimodal input data is required before broader workflow integration can be recommended.