{"title":"Learning region-aware style-content feature transformations for face image beautification","authors":"Zhen Xu, Si Wu","doi":"10.1016/j.patcog.2025.111861","DOIUrl":null,"url":null,"abstract":"<div><div>As a representative image-to-image translation task, facial makeup transfer is typically performed by applying intermediate feature normalization, conditioned on the style information extracted from a reference image. However, the relevant methods are typically limited in range of applicability, due to that the style information is independent of source images and lack of spatial details. To realize precise makeup transfer and further associate with face component editing, we propose a Semantic Region Style-content Feature Transformation approach, which is referred to as SRSFT. Specifically, we encode both reference and source images into region-wise feature vectors and maps, based on semantic segmentation masks. To address the misalignment in poses and expressions, region-wise spatial transformations are inferred to align the reference and source masks, and are then applied to explicitly warp the reference feature maps to the source face, without any extra supervision. The resulting feature maps are fused with the source ones and inserted into a generator for image synthesis. On the other hand, the reference and source feature vectors are also fused and used to determine the modulation parameters at multiple intermediate layers. SRSFT is able to achieve superior beautification performance in terms of seamlessness and fidelity.</div></div>","PeriodicalId":49713,"journal":{"name":"Pattern Recognition","volume":"170 ","pages":"Article 111861"},"PeriodicalIF":7.5000,"publicationDate":"2025-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Pattern Recognition","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0031320325005217","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0
Abstract
As a representative image-to-image translation task, facial makeup transfer is typically performed by applying intermediate feature normalization, conditioned on the style information extracted from a reference image. However, the relevant methods are typically limited in range of applicability, due to that the style information is independent of source images and lack of spatial details. To realize precise makeup transfer and further associate with face component editing, we propose a Semantic Region Style-content Feature Transformation approach, which is referred to as SRSFT. Specifically, we encode both reference and source images into region-wise feature vectors and maps, based on semantic segmentation masks. To address the misalignment in poses and expressions, region-wise spatial transformations are inferred to align the reference and source masks, and are then applied to explicitly warp the reference feature maps to the source face, without any extra supervision. The resulting feature maps are fused with the source ones and inserted into a generator for image synthesis. On the other hand, the reference and source feature vectors are also fused and used to determine the modulation parameters at multiple intermediate layers. SRSFT is able to achieve superior beautification performance in terms of seamlessness and fidelity.
期刊介绍:
The field of Pattern Recognition is both mature and rapidly evolving, playing a crucial role in various related fields such as computer vision, image processing, text analysis, and neural networks. It closely intersects with machine learning and is being applied in emerging areas like biometrics, bioinformatics, multimedia data analysis, and data science. The journal Pattern Recognition, established half a century ago during the early days of computer science, has since grown significantly in scope and influence.