{"title":"面部纹理感知器:用输入级诱导偏置感知器实现高保真的面部纹理恢复","authors":"Seungeun Lee","doi":"10.1109/ICASSP49357.2023.10096776","DOIUrl":null,"url":null,"abstract":"This paper presents a new method, called Facial Texture Perceiver. It deals with the task of facial texture recovery from in-the-wild images without 3D supervision. Motivated by their success in various computer vision tasks, we attempt to use transformers for this task. However, capturing high-fidelity facial details requires a large number of mesh vertices and in this case, naively applying vanilla transformer can incur prohibitively high computational and memory costs. We address this challenge by mapping the input with a large number of mesh vertices to a latent space and performing their attention on this space. Also, we introduce input-level inductive biases by injecting the geometry and appearance embeddings as extra inputs. It helps to data-efficiently learn and generalize in-the-wild domains. The resulting architecture enable the application of Transformers to high-resolution facial meshes. Experiments on CelebA, MICC-Florence and MoFA-test datasets demonstrate that our method can accurately reconstruct facial textures, outperforming state-of-the-art methods.","PeriodicalId":113072,"journal":{"name":"ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","volume":"58 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2023-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Facial Texure Perceiver: Towards High-Fidelity Facial Texture Recovery with Input-Level Inductive Biased Perceiver IO\",\"authors\":\"Seungeun Lee\",\"doi\":\"10.1109/ICASSP49357.2023.10096776\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"This paper presents a new method, called Facial Texture Perceiver. It deals with the task of facial texture recovery from in-the-wild images without 3D supervision. Motivated by their success in various computer vision tasks, we attempt to use transformers for this task. However, capturing high-fidelity facial details requires a large number of mesh vertices and in this case, naively applying vanilla transformer can incur prohibitively high computational and memory costs. We address this challenge by mapping the input with a large number of mesh vertices to a latent space and performing their attention on this space. Also, we introduce input-level inductive biases by injecting the geometry and appearance embeddings as extra inputs. It helps to data-efficiently learn and generalize in-the-wild domains. The resulting architecture enable the application of Transformers to high-resolution facial meshes. Experiments on CelebA, MICC-Florence and MoFA-test datasets demonstrate that our method can accurately reconstruct facial textures, outperforming state-of-the-art methods.\",\"PeriodicalId\":113072,\"journal\":{\"name\":\"ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)\",\"volume\":\"58 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2023-06-04\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/ICASSP49357.2023.10096776\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/ICASSP49357.2023.10096776","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
Facial Texure Perceiver: Towards High-Fidelity Facial Texture Recovery with Input-Level Inductive Biased Perceiver IO
This paper presents a new method, called Facial Texture Perceiver. It deals with the task of facial texture recovery from in-the-wild images without 3D supervision. Motivated by their success in various computer vision tasks, we attempt to use transformers for this task. However, capturing high-fidelity facial details requires a large number of mesh vertices and in this case, naively applying vanilla transformer can incur prohibitively high computational and memory costs. We address this challenge by mapping the input with a large number of mesh vertices to a latent space and performing their attention on this space. Also, we introduce input-level inductive biases by injecting the geometry and appearance embeddings as extra inputs. It helps to data-efficiently learn and generalize in-the-wild domains. The resulting architecture enable the application of Transformers to high-resolution facial meshes. Experiments on CelebA, MICC-Florence and MoFA-test datasets demonstrate that our method can accurately reconstruct facial textures, outperforming state-of-the-art methods.