Spectrum intervention based invariant causal representation learning for single-domain generalizable medical image segmentation

Abstract

The performance of a well-trained segmentation model is often trapped by domain shift caused by acquisition variance. Existing efforts are devoted to expanding the diversity of single-source samples, as well as learning domain-invariant representations. Essentially, they are still modeling the statistical dependence between sample-label pairs to achieve a superficial portrayal of reality. On the contrary, we propose a Spectrum Intervention based Invariant Causal Representation Learning (SI${}^2$CRL) framework, to unify the data generation and representation learning from causal view. Specifically, for the data generation, the unknown object elements can be reified in frequency domain as phase variables, then we propose an amplitude-based intervention module to generate low-frequency perturbations via random-weighted multilayer convolutional network. For the causal representations, a two-stage causal synergy modeling process is proposed to derive unobservable causal factors. In the first stage, the style-sensitive non-causal factors lying in the shallow layer of encoder are filtered out by contrastive-based causal decoupling mechanism. In the second stage, the hierarchical features in decoder are first factorized with cross-covariance regularization to ensure channel-wise independence; Subsequently, we introduce an adversarial-based causal purification module, which encourages the decoder to iteratively update causally sufficient information and make domain-robust predictions. We evaluate our SI${}^2$CRL against the state-of-the-art methods on cross-site prostate MRI segmentation, cross-modality (CT-MRI) abdominal multi-organ segmentation, and cross-sequence (MRI) cardiac segmentation. Our approach achieves consistent performance gains compared to these peer methods.