Rethinking hard training sample generation for medical image segmentation

Abstract

This paper tackles the task of synthetic data generation for downstream segmentation tasks, especially in data-scarce fields like medical diagnostics. Previous methods address the challenge of similar synthetic samples leading to model saturation by leveraging the specific downstream model to guide the generation process, and dynamically adjusting sample difficulty to prevent downstream performance plateaus. However, such an approach never considers the interoperability of these synthetic samples, which may not be universally challenging due to varying feature focuses across different downstream models. Thus, we propose a strategy that uses the discrepancy between backbone-extracted features and real image prototypes to generate challenging samples, employing two loss functions: one for key-area diversity and another for overall image fidelity. This ensures key areas are challenging while the background remains stable, creating samples that are broadly applicable for downstream tasks without overfitting to specific models. Our method, leveraging the data generated by our approach for model training, achieves an average mean Intersection over Union (mIoU) of 86.84% across five polyp test datasets, surpassing the state-of-the-art (SOTA) model CTNet [1] by a significant margin of 6.14%. Code is available at https://github.com/Bbinzz/Rethinking-Hard-Training-Sample-Generation-for-Medical-Image-Segmentation.