EnsembleSleepNet: a novel ensemble deep learning model based on transformers and attention mechanisms using multimodal data for sleep stages classification

Abstract

Classifying sleep stages using biological signals is an important and challenging task in sleep medicine. Combining deep learning networks with transformers and attention mechanisms represents a powerful approach for achieving high-performance results in classification tasks. Multimodal learning, which integrates various types of input data, can significantly enhance the classification performance of these networks. However, many existing studies either rely on single-modal data or design a single model to handle different signals and modalities without considering the unique characteristics of each data type, which often fails to capture optimal features. To address this limitation, we propose an ensemble model for sleep stage classification that leverages multimodal data, including raw signals, spectrograms, and handcrafted features. We utilize the Sleep Heart Health Study (SHHS) dataset by selecting multiple signals from polysomnography recordings. Our approach develops three specialized sub-models with different layers and components, each designed based on the unique characteristics of specific data types and signals, and integrates them into a unified ensemble deep learning framework. The proposed EnsembleSleepNet achieved comparable performance against existing methods by obtaining high values of 0.897, 0.852, and 0.831 in accuracy, Cohen's kappa (κ), and Macro F1 score (MF1) respectively. Additionally, ablation studies revealed the impact of the selected signals and components in our developed model.