ArtiDiffuser: A unified framework for artifact restoration and synthesis for histology images via counterfactual diffusion model

Abstract

Artifacts in histology images pose challenges for accurate diagnosis with deep learning models, often leading to misinterpretations. Existing artifact restoration methods primarily rely on Generative Adversarial Networks (GANs), which approach the problem as image-to-image translation. However, those approaches are prone to mode collapse and can unexpectedly alter morphological features or staining styles. To address the issue, we propose ArtiDiffuser, a counterfactual diffusion model tailored to restore only artifact-distorted regions while preserving the integrity of the rest of the image. Additionally, we show an innovative perspective by addressing the misdiagnosis stemming from artifacts via artifact synthesis as data augmentation, and thereby leverage ArtiDiffuser to unify the artifact synthesis and the restoration capabilities. This synergy significantly surpasses the performance of conventional methods which separately handle artifact restoration or synthesis. We propose a Swin-Transformer denoising network backbone to capture both local and global attention, further enhanced with a class-guided Mixture of Experts (MoE) to process features related to specific artifact categories. Moreover, it utilizes adaptable class-specific tokens for enhanced feature discrimination and a mask-weighted loss function to specifically target and correct artifact-affected regions, thus addressing issues of data imbalance. In downstream applications, ArtiDiffuser employs a consistency regularization strategy that assures the model’s predictive accuracy is maintained across original and artifact-augmented images. We also contribute the first comprehensive histology dataset, comprising 723 annotated patches across various artifact categories, to facilitate further research. Evaluations on four distinct datasets for both restoration and synthesis demonstrate ArtiDiffuser’s effectiveness compared to GAN-based approaches, used for either pre-processing or augmentation. The code is available at https://github.com/wagnchogn/ArtiDiffuser.