Dual-Stream Hybrid Network Based on Global and Local Spectral Fusion for Decoding EEG and sEMG Fusion Signals

Abstract

Electroencephalography (EEG) and surface electromyography (sEMG) play a crucial role in capturing the central motor nervous system’s activities, thereby serving as vital tools in the realms of rehabilitation and assistive control for individuals with neurological disorders. Nonetheless, the reliance on a singular signal modality for action classification is fraught with challenges, ranging from limited accuracy, diminished interference resilience to susceptibility to muscle fatigue. The hybrid brain-computer interface (hBCI) integrating EEG and sEMG signals, synergistically harness the strengths of both signals. Our innovative approach integrates the short-time Fourier transform-based global with local spectral feature fusion (STFT-GLSF) method to elucidate the interrelationship between EEG and sEMG signals. This method utilizes a dual-fusion strategy, effectively capturing both the pivotal and overarching features within the signals. Furthermore, we have developed an advanced dual-stream hybrid residual network, AC-DSHResNet, which simultaneously utilizes attention mechanisms with ConvLSTM. This model’s dual-branch architecture is specifically engineered to refine feature representation in motion decoding. Rigorous validation on both lab-collected and publicly available datasets substantiates the efficacy of our method, achieving an impressive 95.39% accuracy on lab datasets and 88% on public datasets. Compared to existing decoding techniques, our proposed model demonstrates superior performance. These results unequivocally demonstrate the versatility and effectiveness of our model in accurately classifying actions across diverse tasks and experimental paradigms, thereby significantly enhancing the reliability and effectiveness of neurological disease rehabilitation training through the strategic fusion of EEG and sEMG signals.