Multi-modal Fundus Image Registration with Deep Feature Matching and Image Scaling

Abstract

Multi-modal image registration is a technology that converts heterogeneous spatial coordinate systems of different images into one unified coordinate system. This is a fundamental task of medical image analysis as well as computer vision domain because it facilitates a comprehensive understanding of images captured by aligning two or more images. In the field of ophthalmology, the registration is beneficial for ophthalmologists in clinical trial diagnosis, plan treatment, and image-guided surgery. However, registering for two types of fundus image, including conventional and ultra-wide-field fundus images, is a challenging task due to the highly difference in scales of the images. The paper proposes a method of scaling and register images based on common features (e.g., optic disc) of the two fundus image types taken from the same patient. This method improves the performance of multi-modal image registration by reducing the distance in the homogeneous coordinate system of two images through image scaling. The proposed method improves about 13% correct keypoints compared to the conventional deep learning method.