MaskGAN: a virtual photoacoustic histological staining method with interest region compensation based on deep learning

Abstract

Ultraviolet photoacoustic microscopy (UV-PAM) enables the visualization of cell nuclei by leveraging the endogenous optical absorption contrast of DNA and RNA, eliminating the need for complex and time-consuming sample preparation steps such as staining. However, photoacoustic histological images are typically presented as grayscale maps based on photoacoustic signal intensity, which differ substantially from the hematoxylin and eosin (H&E) stained images that pathologists are accustomed to interpreting. This mismatch limits the clinical translation of photoacoustic imaging in pathological applications. To address this challenge, we propose a virtual staining method for photoacoustic histological imaging based on a region-of-interest (ROI) compensation mechanism. The method employs a prior-guided weakly supervised model built upon a cycle-consistent network architecture, with an additional constraint using binary cross-entropy loss applied to nuclear masks of real and generated images. This design enables the model to focus on learning biologically meaningful latent mappings while suppressing non-essential information such as image noise. Experimental results show that the introduction of the ROI compensation mechanism leads to significant improvements in multi-dimensional image quality metrics across network depths ranging from 10 to 22 layers. Moreover, it yields lower quantification errors in nuclear features. Compared to state-of-the-art methods, MaskGAN achieves the best performance across all evaluated metrics, demonstrating superior capabilities in virtual staining and intra-nuclear detail preservation. This virtual staining approach effectively circumvents the need for labor-intensive clinical histological staining procedures, offering promising potential for clinical translation. The full networks are available at https://github.com/tianhe6HH/MaskGAN/tree/master.