Information-Theoretic Analysis of Multimodal Image Translation

Abstract

Multimodal image translation has found useful applications in solving several medical imaging problems. In this paper, we presented a systematic analysis of multimodal images and machine learning-based image translation from an information-theoretic perspective. Specifically, we analyzed the amount of mutual information that exists in some commonly used multimodal images. This analysis revealed varying structural correlation across modalities and tissue-dependence of mutual information. We also analyzed the amount of information transferred and gained in multimodal image translation and provided an upper bound on the information gain. Information-theoretic measures were also proposed to assess the effectiveness of an image translator, and the uncertainty associated with image translation. Numerical results were presented to demonstrate the information gain in practical multimodal image translation, and to validate the proposed upper bound on information gain and the translation error predictor. Finally, several potential applications of our analysis results were discussed, including the image denoising and reconstruction using side information generated by image translation. The findings from this study may prove useful for guiding the further development and application of multimodal image translation.