Ning Huang, Zhengtao Xi, Yingying Jiao, Yudong Zhang, Zhuqing Jiao, Xiaona Li
{"title":"Multi-modal feature fusion with multi-head self-attention for epileptic EEG signals.","authors":"Ning Huang, Zhengtao Xi, Yingying Jiao, Yudong Zhang, Zhuqing Jiao, Xiaona Li","doi":"10.3934/mbe.2024304","DOIUrl":null,"url":null,"abstract":"<p><p>It is important to classify electroencephalography (EEG) signals automatically for the diagnosis and treatment of epilepsy. Currently, the dominant single-modal feature extraction methods cannot cover the information of different modalities, resulting in poor classification performance of existing methods, especially the multi-classification problem. We proposed a multi-modal feature fusion (MMFF) method for epileptic EEG signals. First, the time domain features were extracted by kernel principal component analysis, the frequency domain features were extracted by short-time Fourier extracted transform, and the nonlinear dynamic features were extracted by calculating sample entropy. On this basis, the features of these three modalities were interactively learned through the multi-head self-attention mechanism, and the attention weights were trained simultaneously. The fused features were obtained by combining the value vectors of feature representations, while the time, frequency, and nonlinear dynamics information were retained to screen out more representative epileptic features and improve the accuracy of feature extraction. Finally, the feature fusion method was applied to epileptic EEG signal classifications. The experimental results demonstrated that the proposed method achieves a classification accuracy of 92.76 ± 1.64% across the five-category classification task for epileptic EEG signals. The multi-head self-attention mechanism promotes the fusion of multi-modal features and offers an efficient and novel approach for diagnosing and treating epilepsy.</p>","PeriodicalId":49870,"journal":{"name":"Mathematical Biosciences and Engineering","volume":"21 8","pages":"6918-6935"},"PeriodicalIF":2.6000,"publicationDate":"2024-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Mathematical Biosciences and Engineering","FirstCategoryId":"5","ListUrlMain":"https://doi.org/10.3934/mbe.2024304","RegionNum":4,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"Mathematics","Score":null,"Total":0}
引用次数: 0
Abstract
It is important to classify electroencephalography (EEG) signals automatically for the diagnosis and treatment of epilepsy. Currently, the dominant single-modal feature extraction methods cannot cover the information of different modalities, resulting in poor classification performance of existing methods, especially the multi-classification problem. We proposed a multi-modal feature fusion (MMFF) method for epileptic EEG signals. First, the time domain features were extracted by kernel principal component analysis, the frequency domain features were extracted by short-time Fourier extracted transform, and the nonlinear dynamic features were extracted by calculating sample entropy. On this basis, the features of these three modalities were interactively learned through the multi-head self-attention mechanism, and the attention weights were trained simultaneously. The fused features were obtained by combining the value vectors of feature representations, while the time, frequency, and nonlinear dynamics information were retained to screen out more representative epileptic features and improve the accuracy of feature extraction. Finally, the feature fusion method was applied to epileptic EEG signal classifications. The experimental results demonstrated that the proposed method achieves a classification accuracy of 92.76 ± 1.64% across the five-category classification task for epileptic EEG signals. The multi-head self-attention mechanism promotes the fusion of multi-modal features and offers an efficient and novel approach for diagnosing and treating epilepsy.
期刊介绍:
Mathematical Biosciences and Engineering (MBE) is an interdisciplinary Open Access journal promoting cutting-edge research, technology transfer and knowledge translation about complex data and information processing.
MBE publishes Research articles (long and original research); Communications (short and novel research); Expository papers; Technology Transfer and Knowledge Translation reports (description of new technologies and products); Announcements and Industrial Progress and News (announcements and even advertisement, including major conferences).