Meng Tang , Pengrui Li , Haokai Zhang , Liu Deng , Shihong Liu , Qingyuan Zheng , Hongli Chang , Changming Zhao , Manqing Wang , Guilai Zuo , Dongrui Gao
{"title":"HMS-TENet:基于脑电图和眼电图的分层多尺度拓扑增强网络,用于驾驶员警惕性估计","authors":"Meng Tang , Pengrui Li , Haokai Zhang , Liu Deng , Shihong Liu , Qingyuan Zheng , Hongli Chang , Changming Zhao , Manqing Wang , Guilai Zuo , Dongrui Gao","doi":"10.1016/j.bmt.2024.10.003","DOIUrl":null,"url":null,"abstract":"<div><div>Driving vigilance estimation is an important task for traffic safety. Nowadays, electroencephalography (EEG) and electrooculography (EOG) have made some achievements in vigilance estimation, but there are still some challenges: 1) The traditional approachs with direct multimodal fusion may face the problems of information redundancy and data dimensionality mismatch; 2) Capture key discriminative features during multimodal fusion without losing specific patterns to each modality. In order to solve the above problems, this paper proposes a approach with fusion of EEG and EOG features in split bands, which not only preserves the information about brain activities in different bands of EEG, but also effectively integrates the relevant information of EOG. On this basis, we further propose a hierarchical multi-scale topological enhanced network (HMS-TENet). This network first introduces a pyramid pooling structure (PPS) to capture contextual relationships from different discriminative perspectives. And then we design a selective convolutional structure (SCS) for adaptive sense-field selection, which enables us to mine the desired discriminative information in small-size features. In addition, we design a topology self-aware attention to enhance the learning of representations of complex topological relationships among EEG channels. Finally, the output of the model can be selected for both regression and classification tasks, providing higher flexibility and adaptability. We demonstrate the robustness, generalizability, and utility of the proposed method based on intra-subject and cross-subject experiments on the SEED-VIG public dataset. Codes are available at <span><span>https://github.com/tangmeng28/HMS-TENet</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":100180,"journal":{"name":"Biomedical Technology","volume":"8 ","pages":"Pages 92-103"},"PeriodicalIF":0.0000,"publicationDate":"2024-10-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"HMS-TENet: A hierarchical multi-scale topological enhanced network based on EEG and EOG for driver vigilance estimation\",\"authors\":\"Meng Tang , Pengrui Li , Haokai Zhang , Liu Deng , Shihong Liu , Qingyuan Zheng , Hongli Chang , Changming Zhao , Manqing Wang , Guilai Zuo , Dongrui Gao\",\"doi\":\"10.1016/j.bmt.2024.10.003\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><div>Driving vigilance estimation is an important task for traffic safety. Nowadays, electroencephalography (EEG) and electrooculography (EOG) have made some achievements in vigilance estimation, but there are still some challenges: 1) The traditional approachs with direct multimodal fusion may face the problems of information redundancy and data dimensionality mismatch; 2) Capture key discriminative features during multimodal fusion without losing specific patterns to each modality. In order to solve the above problems, this paper proposes a approach with fusion of EEG and EOG features in split bands, which not only preserves the information about brain activities in different bands of EEG, but also effectively integrates the relevant information of EOG. On this basis, we further propose a hierarchical multi-scale topological enhanced network (HMS-TENet). This network first introduces a pyramid pooling structure (PPS) to capture contextual relationships from different discriminative perspectives. And then we design a selective convolutional structure (SCS) for adaptive sense-field selection, which enables us to mine the desired discriminative information in small-size features. In addition, we design a topology self-aware attention to enhance the learning of representations of complex topological relationships among EEG channels. Finally, the output of the model can be selected for both regression and classification tasks, providing higher flexibility and adaptability. We demonstrate the robustness, generalizability, and utility of the proposed method based on intra-subject and cross-subject experiments on the SEED-VIG public dataset. Codes are available at <span><span>https://github.com/tangmeng28/HMS-TENet</span><svg><path></path></svg></span>.</div></div>\",\"PeriodicalId\":100180,\"journal\":{\"name\":\"Biomedical Technology\",\"volume\":\"8 \",\"pages\":\"Pages 92-103\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-10-28\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Biomedical Technology\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S2949723X24000333\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Biomedical Technology","FirstCategoryId":"1085","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2949723X24000333","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
HMS-TENet: A hierarchical multi-scale topological enhanced network based on EEG and EOG for driver vigilance estimation
Driving vigilance estimation is an important task for traffic safety. Nowadays, electroencephalography (EEG) and electrooculography (EOG) have made some achievements in vigilance estimation, but there are still some challenges: 1) The traditional approachs with direct multimodal fusion may face the problems of information redundancy and data dimensionality mismatch; 2) Capture key discriminative features during multimodal fusion without losing specific patterns to each modality. In order to solve the above problems, this paper proposes a approach with fusion of EEG and EOG features in split bands, which not only preserves the information about brain activities in different bands of EEG, but also effectively integrates the relevant information of EOG. On this basis, we further propose a hierarchical multi-scale topological enhanced network (HMS-TENet). This network first introduces a pyramid pooling structure (PPS) to capture contextual relationships from different discriminative perspectives. And then we design a selective convolutional structure (SCS) for adaptive sense-field selection, which enables us to mine the desired discriminative information in small-size features. In addition, we design a topology self-aware attention to enhance the learning of representations of complex topological relationships among EEG channels. Finally, the output of the model can be selected for both regression and classification tasks, providing higher flexibility and adaptability. We demonstrate the robustness, generalizability, and utility of the proposed method based on intra-subject and cross-subject experiments on the SEED-VIG public dataset. Codes are available at https://github.com/tangmeng28/HMS-TENet.