CeRF: Convolutional neural radiance derivative fields for new view synthesis

Abstract

Recently, Neural Radiance Fields (NeRF) has seen a surge in popularity, driven by its ability to generate high-fidelity novel view synthesized images. However, unexpected “floating ghost” artifacts usually emerge with limited training views and intricate optical phenomena. This issue stems from the inherent ambiguities in radiance fields, rooted in the fundamental volume rendering equation and the unrestricted learning paradigms in multi-layer perceptrons. In this paper, we introduce Convolutional Neural Radiance Fields (CeRF), a novel approach to model the derivatives of radiance along rays and solve the ambiguities through a fully neural rendering pipeline. To this end, a single-surface selection mechanism involving both a modified softmax function and an ideal point is proposed to implement our radiance derivative fields. Furthermore, a structured neural network architecture with 1D convolutional operations is employed to further boost the performance by extracting latent ray representations. Extensive experiments demonstrate the promising results of our proposed model compared with existing state-of-the-art approaches.