Synergizing ECG and textual features: A multi-modal method for coronary artery disease classification

Abstract

Exercise Stress Test (EST) is a non-invasive method widely used to diagnose Coronary Artery Disease (CAD). It generates a substantial amount of multi-modal data, which is crucial for diagnostic research. However, previous research has tended to ignore textual modalities such as patients’ subjective symptoms and clinicians’ interpretations during the diagnostic process. This research aims to improve CAD diagnosis by combining multi-modal data, including patients’ ECG, physiological data, subjective symptoms, and clinicians’ notes. This research used data from 404 patients who underwent EST. After preprocessing, a multi-modal model was developed to distinguish CAD negative, CAD positive, and suspected CAD cases. This model consisted of a Time-aware convolutional network (TACN) for extracting temporal features from ECG images and a fine-tuned pre-trained BERT model for textual modality. The proposed method demonstrated strong performance in accuracy, sensitivity, specificity, and positive predictive value, achieving macro-average scores of 87.43%, 77.06%, 91.13%, and 73.89%, respectively. This research proposes a multi-modal model that combines TACN and a fine-tuned BERT model, and improves the model’s classification accuracy for CAD by introducing textual modalities and improving ECG processing. The implementation helps clinicians diagnose CAD more accurately and better allocate medical resources.