Fast Reconstruction of Monocular Human Video Based on KAN

Abstract

Creating 3-D digital people from monocular video provides many possibilities for a wide range of users and rich applications. In this article, we propose a fast, high-quality, and effective method for creating 3-D digital humans from monocular videos, achieving fast training (2.5 min) and real-time rendering. Specifically, we use 3-D Gaussian splatting (3DGS), based on the introduction of skinned multiperson linear model (SMPL) human structure prior, and an optimized Kolmogorov-Arnold network (KAN) neural network to build effective posture and linear blend skinning (LBS) weight estimation module to quickly and accurately learn the fine details of the 3-D human body. In addition, to achieve fast optimization in the densification and prune stages, we propose a two-stage optimization method. First, the local 3-D area that needs to be densified is extracted based on LightGlue, and then KL divergence combined with human body prior is further used to guide Gaussian splitting/cloning and merging operations. We conducted extensive experiments on the ZJU_MoCap dataset, and the peak signal-to-noise ratio (PSNR) and learned perceptual image patch similarity (LPIPS) metrics indicate that we effectively improved rendering quality while ensuring rendering speed.