Robustness of Deep Learning Methods for Ocular Fundus Segmentation: Evaluation of Blur Sensitivity

Abstract

This paper analyzes the sensitivity of deep learning methods for ocular fundus segmentation. We use an empirical methodology based on non-adversarial perturbed datasets. The research is motivated by the perceived needs of mass screening and self-administered tests in which autonomous or semi-autonomous artificially intelligent methods are needed and may be given substandard images with focus issues. These substandard pictures are simulated using blurring algorithms of varying designs and kernel sizes which are subjected to a test of inter-network sensitivity. The network's result on an unblurred original is derived from the testing subset of the DRIVE ocular fundus image dataset used as the ground truth. The networks studied were VesselUNet (Ronnenberger et al. and Huang et al.), VesselGAN (Son et al.), and VesselFCNN (Oliveira et al.). Statistical analysis of the resultant n = 3600 sample has determined that the datapoints indicating sensitivity over kernel size can be fitted with a sigmoid (with a maximum final tolerance of 9.33e-6), and that it can be shown, using robust pairwise Holm-corrected comparisons, that VesselUNet is the least sensitive (with p-values <5e-8). The least disruptive was Gaussian blur, and the most disruptive motion blur unaligned with cardinal axes. The analysis gives us cause to believe that there is reason to research this problem with more depth, and to work on developing more robust methods for ocular fundus segmentation.