Peng Yu , Xiaopeng He , Haoyu Li , Haowen Dou , Yeyu Tan , Hao Wu , Badong Chen
{"title":"FMLAN: A novel framework for cross-subject and cross-session EEG emotion recognition","authors":"Peng Yu , Xiaopeng He , Haoyu Li , Haowen Dou , Yeyu Tan , Hao Wu , Badong Chen","doi":"10.1016/j.bspc.2024.106912","DOIUrl":null,"url":null,"abstract":"<div><div>Emotion recognition is significant in brain-computer interface (BCI) applications. Electroencephalography (EEG) is extensively employed for emotion recognition because of its precise temporal resolution and dependability. However, EEG signals are variable across subjects and sessions, limiting the effectiveness of emotion recognition methods on new users. To address this problem, multi-source domain adaptation was introduced to EEG emotion recognition. Actually, for cross-subject and cross-session emotion recognition methods, there are two most important aspects: extracting features relevant to the emotion recognition task and aligning the features of labeled subjects or sessions(source domains) with those of the unlabeled subject or session(target domain). In this study, we propose a Fine-grained Mutual Learning Adaptation Network (FMLAN) to make innovative improvements in these two aspects. Specifically, we establish multiple separate domain adaptation sub-networks, each corresponding to a specific source domain. Additionally, we introduce a single joint domain adaptation sub-network that combines multiple source domains together. For EEG emotion recognition, we introduce mutual learning for the first time to connect separate domain adaptation networks and joint domain adaptation sub-network. This facilitates the transfer of complementary information between different domains, enabling each sub-network to extract more comprehensive and robust features. Additionally, we design a novel Fine-grained Alignment Module (FAM), which takes category and decision boundary information into account during the feature alignment, ensuring more accurate alignment. Extensive experiments on SEED and SEED-IV datasets demonstrate that our approach outperforms state-of-the-art methods in performance.</div></div>","PeriodicalId":55362,"journal":{"name":"Biomedical Signal Processing and Control","volume":"100 ","pages":"Article 106912"},"PeriodicalIF":4.9000,"publicationDate":"2024-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Biomedical Signal Processing and Control","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1746809424009704","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, BIOMEDICAL","Score":null,"Total":0}
引用次数: 0
Abstract
Emotion recognition is significant in brain-computer interface (BCI) applications. Electroencephalography (EEG) is extensively employed for emotion recognition because of its precise temporal resolution and dependability. However, EEG signals are variable across subjects and sessions, limiting the effectiveness of emotion recognition methods on new users. To address this problem, multi-source domain adaptation was introduced to EEG emotion recognition. Actually, for cross-subject and cross-session emotion recognition methods, there are two most important aspects: extracting features relevant to the emotion recognition task and aligning the features of labeled subjects or sessions(source domains) with those of the unlabeled subject or session(target domain). In this study, we propose a Fine-grained Mutual Learning Adaptation Network (FMLAN) to make innovative improvements in these two aspects. Specifically, we establish multiple separate domain adaptation sub-networks, each corresponding to a specific source domain. Additionally, we introduce a single joint domain adaptation sub-network that combines multiple source domains together. For EEG emotion recognition, we introduce mutual learning for the first time to connect separate domain adaptation networks and joint domain adaptation sub-network. This facilitates the transfer of complementary information between different domains, enabling each sub-network to extract more comprehensive and robust features. Additionally, we design a novel Fine-grained Alignment Module (FAM), which takes category and decision boundary information into account during the feature alignment, ensuring more accurate alignment. Extensive experiments on SEED and SEED-IV datasets demonstrate that our approach outperforms state-of-the-art methods in performance.
期刊介绍:
Biomedical Signal Processing and Control aims to provide a cross-disciplinary international forum for the interchange of information on research in the measurement and analysis of signals and images in clinical medicine and the biological sciences. Emphasis is placed on contributions dealing with the practical, applications-led research on the use of methods and devices in clinical diagnosis, patient monitoring and management.
Biomedical Signal Processing and Control reflects the main areas in which these methods are being used and developed at the interface of both engineering and clinical science. The scope of the journal is defined to include relevant review papers, technical notes, short communications and letters. Tutorial papers and special issues will also be published.