Explainable artificial intelligence-based identification of the localized events in imagined speech electroencephalogram

Abstract

Localizing events in Imagined Speech Electroencephalogram (IS-EEG) signals is considered vital for analyzing significant neural activity that may otherwise be obscured by non-event segments. In this study, a Deep Learning framework is introduced, utilizing a hybrid Long Short-Term Memory-Convolutional Neural Network (LSTM-CNN) for effective IS-EEG signal recognition and event localization. A key component of this framework is the implementation of Gradient-weighted Class Activation Mapping (Grad-CAM) as an explainability technique, which is used to generate heatmaps that highlight critical regions in the IS-EEG signals and validate the model’s classification decisions. The IS-EEG signals were used to train and validate the proposed LSTM-CNN model, which achieved an accuracy of 97.24%. Subsequently, the IS-EEG signals were analyzed to estimate the theoretical duration of localized events, which was found to lie between 0.9 and 1.8 s. The trained model and the derived duration estimates were then utilized to determine an optimal threshold of 0.28 based on the average performance with masking. Furthermore, masking of non-critical segments led to an accuracy improvement to 99.17%, while masking of essential regions resulted in poor performance. The robustness of the model was also evaluated by introducing controlled levels of noise, with the Grad-CAM heatmaps demonstrating reasonable consistency in the presence of noise.