EEG-based schizophrenia detection using fusion of effective connectivity maps and convolutional neural networks with transfer learning

Schizophrenia (SZ) is a serious mental disorder that can mainly be distinguished by symptoms including delusions and hallucinations. This mental disorder makes difficult conditions for the person and her/his relatives. Electroencephalogram (EEG) signal is a sophisticated neuroimaging technique that helps neurologists to diagnose this mental disorder. Estimating and evaluating brain effective connectivity between electrode pairs is an appropriate way of diagnosing brain states in neuroscience studies. In this study, we construct a novel image from multi-channels of EEG based on the fusion of three effective connectivity, partial directed coherence (PDC), and direct directed transfer function (dDTF) and transfer entropy (TE) at three consecutive time windows. Then, this image was used as input of five well-known convolutional neural networks (CNNs) through transfer learning (TL) to learn patterns related to SZ patients to diagnose this disorder from normal participants from two public databases. Also, the majority voting method was used to improve these results based on ensemble results of the five CNNs, i.e., ResNet-50, Inception-v3, DenseNet-201, EfficientNetB0, and NasNet-Mobile. The highest average accuracy, specificity and sensitivity to diagnose SZ patients from healthy participants were obtained using EfficientNetB0 through the Leave-One-Subject-out (LOSO) Cross-Validation criterion equal to 96.67%, 96.23%, 96.82%, 95.15%, 94.42% and 96.28% for the first and second databases, respectively. Also, as we suggested, the ensemble approach of EfficientNetB0, ResNet-50 and NasNet-Mobile increased the accuracy by approximately 3%. Our results show the effectiveness of providing fused images from multichannel EEG signals to the ensemble of CNNs through TL to diagnose SZ than state-of-the-art studies.