Evaluating Posterior Vitreous Detachment Annotation Consistency on OCT Scans in Patients with Disease of the Vitreomacular Interface.

Abstract

Purpose: To evaluate inter-grader variability in posterior vitreous detachment (PVD) classification in patients with epiretinal membrane (ERM) and macular hole (MH) on spectral-domain optical coherence tomography (SD-OCT) and identify challenges in defining a reliable ground truth for artificial intelligence (AI)-based tools.

Methods: A total of 437 horizontal SD-OCT B-scans were retrospectively selected and independently annotated by six experienced ophthalmologists adopting four categories: 'full PVD', 'partial PVD', 'no PVD', and 'ungradable'. Inter-grader agreement was assessed using pairwise Cohen's kappa scores. Consensus levels, accuracy, recall, specificity, and grading time were also analyzed using the majority vote as reference.

Results: The overall average Cohen's kappa was 0.57. Agreement was highest for 'partial PVD' (Cohen's kappa = 0.70), followed by 'full PVD' (Cohen's kappa = 0.65), and lowest for 'no PVD' (Cohen's kappa = 0.14), indicating substantial diagnostic variability. Full consensus was achieved in only 31.1% of OCT scans, while 11.4% required adjudication. The sensitivity for 'no PVD' was notably low (0.35 ± 0.32), and misclassified OCT scans took significantly longer to grade (p < 0.001).

Conclusion: Our results underscore challenges associated with reliable OCT-based classification of PVD in patients with diseases of the vitreomacular interface, especially for cases with completely attached vitreous. Improving inter-grader agreement through consensus grading and advanced imaging modalities will be critical for establishing a solid ground truth to support reliable, AI-driven PVD detection systems.