Fusing Simultaneous EEG and fMRI Using Functional and Anatomical Information

Abstract

Simultaneously measuring electro physical and hemodynamic signals has become more accessible in the last years and the need for modeling techniques that can fuse the modalities is growing. In this work we augment a specific fusion method, the multimodal Source Power Co-modulation (mSPoC), to not only use functional but also anatomical information. The goal is to extract correlated source components from electroencephalography (EEG) and functional magnetic resonance imaging (fMRI). Anatomical information enters our proposed extension to mSPoC via the forward model, which relates the activity on cortex level to the EEG sensors. The augmented mSPoC is shown to outperform the original version in realistic simulations where the signal to noise ratio is low or where training epochs are scarce.