Mutual Information Learning-Based End-to-End Fusion Network for Hybrid EEG-fNIRS Brain–Computer Interface

Abstract

Hybrid brain–computer interfaces (BCIs) integrating electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) hold great potential, but effectively fusing their complementary information remains challenging. In this work, we propose a novel end-to-end EEG-fNIRS fusion network, EFMLNet. EFMLNet comprises two personalized feature extractors and a cross-modal mutual information learning module, designed to fully exploit the spatial and temporal characteristics of each modality. This architecture enables efficient extraction and fusion of complementary information from EEG and fNIRS signals. We evaluate EFMLNet through extensive cross-subject experiments on two public BCI datasets, motor imagery (MI) and mental arithmetic (MA), and show that its classification accuracy reaches 76.8% and 76.5%, respectively, surpassing existing fusion methods. These results demonstrate the effectiveness of EFMLNet in improving hybrid BCI performance.