BORDE: Boundary and Sub-Region Denormalization for Semantic Brain Image Synthesis

Abstract

Medical images are often expensive to acquire and offer limited use due to legal issues besides the lack of consistency and availability of image annotations. Thus, the use of medical datasets can be restrictive for training deep learning models. The generation of synthetic images along with their corresponding annotations can therefore aid to solve this issue. In this paper, we propose a novel Generative Adversarial Network (GAN) generator for multimodal semantic image synthesis of brain images based on a novel denormalization block named BOundary and sub-Region DEnormalization (BORDE). The new architecture consists of a decoder generator that allows: (i) an effectively sequential propagation of a-priori semantic information through the generator, (ii) noise injection at different scales to avoid mode-collapse, and (iii) the generation of rich and diverse multimodal synthetic samples along with their contours. Our model generates very realistic and plausible synthetic images that when combined with real data helps to improve the accuracy in brain segmentation tasks. Quantitative and qualitative results on challenging multimodal brain imaging datasets (BraTS 2020 [1] and ISLES 2018 [2]) demonstrate the advantages of our model over existing image-agnostic state-of-the-art techniques, improving segmentation and semantic image synthesis tasks. This allows us to prove the need for more domain-specific techniques in GANs models.