{"title":"Hierarchical Multivariate Representation Learning for Face Sketch Recognition","authors":"Jiahao Zheng;Yu Tang;Anthony Huang;Dapeng Wu","doi":"10.1109/TETCI.2024.3359090","DOIUrl":null,"url":null,"abstract":"Face Sketch Recognition (FSR) is extremely challenging because of the heterogeneous gap between sketches and images. Relying on the ability to generative models, prior generation-based works have dominated FSR for a long time by decomposing FSR into two steps, namely, heterogeneous data synthesis and homogeneous data matching. However, decomposing FSR into two steps introduces noise and uncertainty, and the first step, heterogeneous data synthesis, is an even general and challenging problem. Solving a specific problem requires solving a more general one is to put the cart before the horse. In order to solve FSR smoothly and circumvent the above problems of generation-based methods, we propose a multi-view representation learning (MRL) framework based on Multivariate Loss and Hierarchical Loss (MvHi). Specifically, by using triplet loss as a bridge to connect the augmented representations generated by InfoNCE, we propose Multivariate Loss (Mv) to construct a more robust common feature subspace between sketches and images and directly solve FSR in this subspace. Moreover, Hierarchical Loss (Hi) is proposed to improve the training stability by utilizing the hidden states of the feature extractor. Comprehensive experiments on two commonly used datasets, CUFS and CUFSF, show that the proposed approach outperforms state-of-the-art methods by more than 7%. In addition, visualization experiments show that the proposed approach can extract the common representations among multi-view data compared to the baseline methods.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":null,"pages":null},"PeriodicalIF":5.3000,"publicationDate":"2024-02-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Emerging Topics in Computational Intelligence","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10432989/","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0
Abstract
Face Sketch Recognition (FSR) is extremely challenging because of the heterogeneous gap between sketches and images. Relying on the ability to generative models, prior generation-based works have dominated FSR for a long time by decomposing FSR into two steps, namely, heterogeneous data synthesis and homogeneous data matching. However, decomposing FSR into two steps introduces noise and uncertainty, and the first step, heterogeneous data synthesis, is an even general and challenging problem. Solving a specific problem requires solving a more general one is to put the cart before the horse. In order to solve FSR smoothly and circumvent the above problems of generation-based methods, we propose a multi-view representation learning (MRL) framework based on Multivariate Loss and Hierarchical Loss (MvHi). Specifically, by using triplet loss as a bridge to connect the augmented representations generated by InfoNCE, we propose Multivariate Loss (Mv) to construct a more robust common feature subspace between sketches and images and directly solve FSR in this subspace. Moreover, Hierarchical Loss (Hi) is proposed to improve the training stability by utilizing the hidden states of the feature extractor. Comprehensive experiments on two commonly used datasets, CUFS and CUFSF, show that the proposed approach outperforms state-of-the-art methods by more than 7%. In addition, visualization experiments show that the proposed approach can extract the common representations among multi-view data compared to the baseline methods.
期刊介绍:
The IEEE Transactions on Emerging Topics in Computational Intelligence (TETCI) publishes original articles on emerging aspects of computational intelligence, including theory, applications, and surveys.
TETCI is an electronics only publication. TETCI publishes six issues per year.
Authors are encouraged to submit manuscripts in any emerging topic in computational intelligence, especially nature-inspired computing topics not covered by other IEEE Computational Intelligence Society journals. A few such illustrative examples are glial cell networks, computational neuroscience, Brain Computer Interface, ambient intelligence, non-fuzzy computing with words, artificial life, cultural learning, artificial endocrine networks, social reasoning, artificial hormone networks, computational intelligence for the IoT and Smart-X technologies.