{"title":"Emotion knowledge-based fine-grained facial expression recognition","authors":"","doi":"10.1016/j.neucom.2024.128536","DOIUrl":null,"url":null,"abstract":"<div><p>Existing facial expression recognition (FER) techniques rely primarily on seven coarse-grained emotions as emotional labels, which are insufficient to cover the subtle changes in human emotions in the real world. We use 135 fine-grained emotions as emotional benchmarks to address the problem of highly semantically similar fine-grained emotion recognition. In this work, we propose a robust emotion knowledge-based fine-grained (EK-FG) emotion recognition network that captures inter-class relationships and discriminative representations of fine-grained emotions through two prior-based losses: coarse-grained hierarchical loss and fine-grained semantic loss. Specifically, the coarse-grained hierarchical loss obtains a structured semantic representation of fine-grained emotions based on prior knowledge, and captures inter-class relationships through effective category-level push-pull to obtain discriminative representations. The fine-grained semantic loss provides more accurate measurement information for semantic features based on prior knowledge, and enhances the model’s discriminative ability for subtle facial expression differences through regression constraints. Extensive experimental results on the Emo135 dataset demonstrate that EK-FG can effectively overcome the class ambiguity of fine-grained emotion.</p></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":null,"pages":null},"PeriodicalIF":5.5000,"publicationDate":"2024-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Neurocomputing","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0925231224013079","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0
Abstract
Existing facial expression recognition (FER) techniques rely primarily on seven coarse-grained emotions as emotional labels, which are insufficient to cover the subtle changes in human emotions in the real world. We use 135 fine-grained emotions as emotional benchmarks to address the problem of highly semantically similar fine-grained emotion recognition. In this work, we propose a robust emotion knowledge-based fine-grained (EK-FG) emotion recognition network that captures inter-class relationships and discriminative representations of fine-grained emotions through two prior-based losses: coarse-grained hierarchical loss and fine-grained semantic loss. Specifically, the coarse-grained hierarchical loss obtains a structured semantic representation of fine-grained emotions based on prior knowledge, and captures inter-class relationships through effective category-level push-pull to obtain discriminative representations. The fine-grained semantic loss provides more accurate measurement information for semantic features based on prior knowledge, and enhances the model’s discriminative ability for subtle facial expression differences through regression constraints. Extensive experimental results on the Emo135 dataset demonstrate that EK-FG can effectively overcome the class ambiguity of fine-grained emotion.
期刊介绍:
Neurocomputing publishes articles describing recent fundamental contributions in the field of neurocomputing. Neurocomputing theory, practice and applications are the essential topics being covered.