Imagined Speech–EEG Detection Using Multivariate Swarm Sparse Decomposition-Based Joint Time–Frequency Analysis for Intuitive BCI

Abstract

In brain–computer interface (BCI) applications, imagined speech (IMS) decoding based on electroencephalogram (EEG) has established a new neuro-paradigm that offers an intuitive communication tool for physically impaired patients. However, existing IMS–EEG-based BCI systems have introduced difficulties in feasible deployment due to nonstationary EEG signals, suboptimal feature extraction, and limited multiclass scalability. To address these challenges, we have presented a novel approach using the multivariate swarm-sparse decomposition method (MSSDM) for joint time–frequency (JTF) analysis and further developed a feasible end-to-end framework from multichannel IMS–EEG signals for IMS detection. MSSDM employs improved multivariate swarm filtering and sparse spectrum techniques to design optimal filter banks for extracting an ensemble of channel-aligned oscillatory components (CAOCs), significantly enhancing IMS activation-related sub-bands. To enhance channel-aligned information, multivariate JTF images have been constructed using JIF and instantaneous amplitude across channels from the obtained CAOCs. Further, JTF-based deep features (JTFDFs) were computed using different pretrained neural networks and mapped most discriminant features using two well-known feature correlation techniques: Canonical correlation analysis and Hellinger distance-based correlation. The proposed method has been tested on the 5-class BCI competition and 6-class Coretto IMS datasets. The experimental findings on cross-subject and cross-dataset reveal that the novel JTFDF feature-based classification model, MSSDM-SqueezeNet-JTFDF, achieved the highest classification performance against all other existing state-of-the-art methods in IMS recognition. Our introduced EEG–BCI models effectively enhance IMS–EEG patterns across multichannel data and offer great potential for the practical deployment of BCI technologies.