Fusion of MRI and PET Images Using Deep Learning Neural Networks

Positron Emission Tomography (PET) and Magnetic Resonance Imaging (MRI) are the two oldest modalities for the detection of brain tumor. These two images provide complementary information. Physicians have to analyze both the images in order to make a decision. Rather than analyzing two different images, it would be better if these images are combined together as a single image. Image fusion refers to the process of combining two different images into a single image. In this research work, a novel feature based image fusion is performed on both MRI and PET images using Convolutional Neural Network (CNN) by extracting features. Features representing texture, shape, edges and other discontiuites are extracted and are then combined to form the output image. Signal to Noise Ratio (SNR) which provides the information present in the input image (<40 represents usefule information in the image) and entropy (entropy approaching one indicates more information) are used as objective measures. Entropy and SNR are higher for CNN based image fusion than that of Discrete Wavelet Transform (DWT). It implies that information from both the input images is available in the output image.