Heterogeneous Modality Dynamic Trustworthy Fusion Network for Cross-Subject Sleep Stage Classification

Abstract

Sleep stage classification plays a pivotal role in clinical practice, significantly enhancing the efficiency of sleep assessment and assisting in the diagnosis of sleep disorders. However, existing sleep stage classification methods are susceptible to heterogeneous multi-modal sleep data and the distribution drift among subjects. Additionally, sleep is a dynamic process, inherently encompassing uncertainty at both temporal and sample levels, which presents challenges for achieving highly trustworthy identification of sleep stages. To address these issues, we propose a Heterogeneous Modality Dynamic Trustworthy fusion network, called HMDT-Net, for cross-subject sleep stage classification. Specifically, we first develop a heterogeneous modality dynamic trustworthy fusion strategy with self-attention and cross-attention mechanisms. It effectively integrates heterogeneous multi-modal data and adaptively estimates uncertainty across different time windows. Second, our approach incorporates adversarial learning to extract a common representation of features across subjects while enhancing the discriminative power of these features. This mitigates the influence of inter-subject variations on the model. Additionally, we introduce a self-paced learning (SPL) mechanism to mitigate the influence of challenging training samples, identified through alignment loss and domain discrimination loss, on the classification model. Finally, we utilize a multi-layer perceptron (MLP) to classify the fused features extracted. Extensive experiments conducted on three multi-modal sleeping datasets containing EEG and EOG data, namely Sleep-EDF-20, Sleep-EDF-78, and SHHS, demonstrate that our proposed method surpasses several state-of-the-art sleep stage classification techniques.