LiteNeRFAvatar: A lightweight NeRF with local feature learning for dynamic human avatar

Abstract

Creating high-quality dynamic human avatars within acceptable costs remains challenging in computer vision and computer graphics. The neural radiance field (NeRF) has become a fundamental means of generating human avatars due to its success in novel view synthesis. However, the storage-intensive and time-consuming per-scene training due to the transformation and evaluation of massive sampling points constrains its practical applications. In this paper, we introduce a novel lightweight NeRF model, LiteNeRFAvatar, to overcome these limits. To avoid the high-cost backward transformation of the sampling points, LiteNeRFAvatar decomposes the appearance features of clothed humans into multiple local feature spaces and transforms them forward according to human movements. Each local feature space affects a limited local area and is represented by an explicit feature volume created by the tensor decomposition techniques to support fast access. The sampling points retrieve the features based on the relative positions to the local feature spaces. The densities and the colors are then regressed from the aggregated features using a tiny decoder. We also adopt an empty space skipping strategy to further reduce the number of sampling points. Experimental results demonstrate that our LiteNeRFAvatar achieves a satisfactory balance between synthesis quality, training time, rendering speed and parameter size compared to the existing NeRF-based methods. For the demo of our method, please refer to the link on: https://youtu.be/UYfreeHtIZY. The source code will be released after the paper is accepted.