Supervised Information Mining From Weakly Paired Images for Breast IHC Virtual Staining

Abstract

Immunohistochemistry (IHC) examination is essential to determine the tumour subtypes, provide key prognostic factors, and develop personalized treatment plans for breast cancer. However, compared to Hematoxylin and Eosin (H&E) staining, the preparation process of IHC staining is more complex and expensive, which limits its application in clinical practice. Therefore, H&E to IHC stain transfer may be an ideal solution to obtain IHC staining. To ensure high transferring quality, it would be much more desirable to exploit the supervised information between adjacent layer images of the same tissue, which are stained by H&E and IHC stainings, respectively. Nevertheless, adjacent layer tissue images are not accurately paired at the pixel level, which poses significant challenges to network training. To address this problem, we propose a generative adversarial network for breast IHC virtual staining, which contains an optimal transport-based supervised information mining (OT-SIM) mechanism and a pathological correlation-based supervised information mining (PC-SIM) mechanism. The OT-SIM guides the network in mining matching consistency between H&E images and the adjacent layer’s real IHC images, providing as much instance-level supervision as possible. The PC-SIM further explores the consistency between the correlation among virtual IHC images and the correlation among real IHC images, providing batch-level supervision. Extensive experiments show the superiority of our method on two breast tissue benchmark datasets compared to the state-of-the-art methods both quantitatively and qualitatively. The code is available at https://github.com/xianchaoguan/SIM-GAN.