A new multimodal medical image fusion framework using Convolution Neural Networks.

Abstract

Medical image fusion reduces the time required for medical diagnosis by creating a composite image from a set of images belonging to different modalities. This paper introduces a deep learning framework for medical image fusion, optimising the number of convolutional layers and selecting an appropriate activation function. The conducted experiments demonstrate that employing three convolution layers with a swish activation function for the intermediate layers is sufficient to extract the salient features of the input images. The tuned features are fused using element-wise fusion rules to prevent the loss of minute details crucial for medical images. The comprehensive fused image is then reconstructed from these features using another set of three convolutional layers. Experimental results confirm that the proposed methodology outperforms other conventional medical image fusion methods in terms of various metrics and the quality of the fused image.