A new method for 3D face reconstruction using transformers based on action unit features

IF 4.1 3区计算机科学 Q1 COMPUTER SCIENCE, INFORMATION SYSTEMS

ICT Express Pub Date : 2025-06-01 DOI:10.1016/j.icte.2025.04.004

Hyeonjin Kim, Hyukjoon Lee

引用次数: 0

Abstract

We present a novel 3D face reconstruction framework called Facial action unit (AU) feature-based 3D FAce Reconstruction using Transformer (AUFART) that can generate a 3D face model that is responsive to AU activation given a single monocular 2D image to capture expressions. We propose a novel 3D face reconstruction framework, called AUFART (Facial Action Unit Feature-based 3D Face Reconstruction using Transformer), which generates 3D face models responsive to AU activations from a single monocular 2D image, effectively capturing facial expressions. AUFART leverages AU-specific features as well as facial global features to achieve accurate 3D reconstruction of facial expressions using transformers. We also introduce a loss function designed to guide the learning process so that the discrepancy in AU activations between the input and rendered reconstruction is minimized. The proposed framework achieves an average F1 score of 0.39, outperforming state-of-the-art methods.

查看原文本刊更多论文

一种基于动作单元特征的三维人脸重构方法

摘要我们提出了一种新的3D人脸重建框架，称为基于面部动作单元（AU）特征的3D人脸重建，该框架使用Transformer (AUFART)，可以生成响应AU激活的3D人脸模型，给定单个单眼2D图像来捕捉表情。我们提出了一种新的3D人脸重建框架，称为AUFART（基于面部动作单元特征的3D人脸重建使用变压器），它可以从单个单眼2D图像中生成响应AU激活的3D人脸模型，有效地捕捉面部表情。AUFART利用au特定的特征以及面部全局特征，使用变压器实现面部表情的精确3D重建。我们还引入了一个损失函数，用于指导学习过程，以便将输入和呈现重建之间AU激活的差异最小化。该框架的F1平均得分为0.39，优于最先进的方法。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

ICT Express Multiple-

CiteScore

10.20

自引率

1.90%

发文量

167

审稿时长

35 weeks

期刊介绍： The ICT Express journal published by the Korean Institute of Communications and Information Sciences (KICS) is an international, peer-reviewed research publication covering all aspects of information and communication technology. The journal aims to publish research that helps advance the theoretical and practical understanding of ICT convergence, platform technologies, communication networks, and device technologies. The technology advancement in information and communication technology (ICT) sector enables portable devices to be always connected while supporting high data rate, resulting in the recent popularity of smartphones that have a considerable impact in economic and social development.