Needle detection and localisation for robot‐assisted subretinal injection using deep learning

Abstract Subretinal injection is a complicated task for retinal surgeons to operate manually. In this paper we demonstrate a robust framework for needle detection and localisation in robot‐assisted subretinal injection using microscope‐integrated Optical Coherence Tomography with deep learning. Five convolutional neural networks with different architectures were evaluated. The main differences between the architectures are the amount of information they receive at the input layer. When evaluated on ex‐vivo pig eyes, the top performing network successfully detected all needles in the dataset and localised them with an Intersection over Union value of 0.55. The algorithm was evaluated by comparing the depth of the top and bottom edge of the predicted bounding box to the ground truth. This analysis showed that the top edge can be used to predict the depth of the needle with a maximum error of 8.5 μm.