CaliDiff: Multi-rater annotation calibrating diffusion probabilistic model towards medical image segmentation

Abstract

Medical image segmentation is critical for accurate diagnostics and effective treatment planning. Traditional multi-rater labeling strategies, while integrating consensus from multiple experts, often do not fully capture the unique insights of individual raters. Moreover, deep discriminative models that aggregate such expert labels typically embed inherent biases into the segmentation results. To address these issues, we introduce CaliDiff, a novel multi-rater annotation calibration diffusion probabilistic model. This model effectively approximates the joint probability distribution among multiple expert annotations and their corresponding images, fully leveraging diverse expert knowledge while actively refining these annotations to approximate the true underlying distribution closely. CaliDiff operates through a structured multi-stage process: it begins with a shared-parameter inverse diffusion to normalize initial expert biases, followed by Expertness Consistent Alignment to minimize variance among annotations and enhance consistency in high-confidence areas. Additionally, we incorporate a Committee-based Endogenous Knowledge Learning mechanism that uses adversarial soft supervision to simulate a reliable pseudo-ground truth, integrating Cross-Expert Fusion and Implicit Consensus Inference. Extensive experimental evaluations on various medical image segmentation datasets show that CaliDiff not only significantly improves the calibration of annotations but also achieves state-of-the-art performance, thereby enhancing the reliability and objectivity of medical diagnostics.