SRMA-KD: Structured relational multi-scale attention knowledge distillation for effective lightweight cardiac image segmentation

Cardiac image segmentation is essential for accurately extracting structural information of the heart, aiding in precise diagnosis and personalized treatment planning. However, real-time segmentation on medical devices demands computational efficiency that often conflicts with the intensive processing and storage requirements of deep learning algorithms. These algorithms are frequently hindered by their complex models and extensive parameter sets, which limit their feasibility in clinical settings with constrained resources. Meanwhile, the performance of lightweight heart segmentation models still requires enhancement. This study introduces the SRMA-KD framework, a knowledge distillation approach for cardiac image segmentation designed to achieve high accuracy with lightweight models. The framework efficiently transfers semantic feature information and structural knowledge from a teacher model to student model, ensuring effective segmentation within clinical resource limitations. The SRMA-KD framework includes three key modules: the Global Structural Relational Block (GSRB), the Multi-scale Feature Attention Block (MFAB), and the Prediction Difference Transfer Block (PDTB). The GSRB correlates the outputs of the teacher and student networks with the ground truth, transferring structural correlations to enhance the student network's global feature learning. The MFAB enables the student network to learn multi-scale feature extraction from the teacher network, focusing on relevant semantic regions. The PDTB minimizes pixel-level differences between the segmentation images of the teacher and student networks. Our experiments demonstrate that the SRMA-KD framework significantly improves the segmentation accuracy of the student network compared to other medical imaging knowledge distillation methods, highlighting its potential as an effective solution for cardiac image segmentation in resource-limited clinical environments.