Enhancing cardiovascular disease diagnosis through knowledge distillation and hybrid Transformer-CNN-SENet architecture.

Abstract

Worldwide, cardiovascular diseases (CVDs) are the leading cause of mortality, making early detection a critical public health objective. The analysis of electrocardiograms (ECGs) is a fundamental tool for CVD diagnosis, and recent developments in deep learning have greatly enhanced its diagnostic accuracy. A 12-lead ECG dataset provides valuable information for enhancing diagnostic performance, but extracting and revealing these patterns requires complex and hidden models with numerous parameters and high computational demands. In portable and wearable healthcare devices, GPUs with low power are not suitable for deploying these models. The purpose of this study is to address these limitations by presenting a knowledge distillation (KD) framework that transfers knowledge from a high-capacity teacher model (T-CST) to a lightweight student model (S-CS). A forward lead selection strategy is introduced to identify the most informative single lead or optimal small-lead combinations for efficient diagnosis with minimal data input. The teacher model integrates CNN, SENet, and Transformer modules to capture both local and long-range temporal dependencies, enabling it to extract a diverse set of hidden and comprehensive patterns from 12-lead ECG signals, including subtle temporal relationships and intricate local features. The compact student model, consisting of a CNN and SENet, leverages this transferred knowledge to achieve comparable performance with substantially lower computational cost. Experiments on the PTB-XL dataset demonstrate that S-CS attains near-teacher accuracy using only three leads while reducing computation by 63 times, highlighting its promise for efficient, accurate, and accessible CVD detection in portable and wearable healthcare systems.