Synthesis of User-Level 3D Head Avatar from Smartphone Video for Virtual Reality

Abstract

The purpose of an avatar is to intensify the sense of reality in a simulated environment such as a virtual reality system. This is done by creating a replica of a real person in virtual reality, as intricately as possible. Implementation of 3D scanners (i.e., Microsoft Kinect) produced vast opportunities for creating 3D models of celebrities and other people who can afford them. These humanoid 3D models as well as fictional human avatars opened the door of perceiving a more realistic 3D system than before. However, a simplified approach of fully rigged 3D head avatar synthesis may help to expand the application of 3D avatars allowing anyone to feel a personalized experience of their own avatar. In this work, we propose a system of 3D head avatar synthesis at the user-end, using an easily accessible and low-cost sensor (smartphone camera). Our proposed methodology consists of data acquisition of the concerned person as video, feature extraction and matching, image registration, formation of structure form motion and post-processing. Without requiring any training data, our method provides a satisfactory performance based on the experimental results. Additionally, we release a public database for avatar synthesis from smartphone video and introduce a set of attributes to evaluate the synthesized 3D avatars.