Knowledge NeRF: Few-shot novel view synthesis for dynamic articulated objects

Abstract

We introduce Knowledge NeRF, a few-shot framework for novel-view synthesis of dynamic articulated objects. Conventional dynamic-NeRF methods learn a deformation field from long monocular videos, yet they degrade sharply when only sparse observations are available. Our key idea is to reuse a high-quality, pose-specific NeRF as a knowledge base and learn a lightweight projection module for each new pose that maps 3-D points in the current state to their canonical counterparts. By freezing the pretrained radiance field and training only this module with five input images, Knowledge NeRF renders novel views whose fidelity matches a NeRF trained with one hundred images. Experimental results demonstrate the effectiveness of our method in reconstructing dynamic 3D scenes with 5 input images in one state. Knowledge NeRF is a new pipeline and a promising solution for novel view synthesis in dynamic articulated objects. The data and implementation will be publicly available at: https://github.com/RussRobin/Knowledge_NeRF.