Perceptual Quality Assessment for NeRF-Generated Scenes: A Training Reference Metric

Abstract

Perceptual quality assessment is a key challenge in traditional image processing as well as emerging AI-based novel view synthesis methods such as neural radiation fields (NeRF). NeRF has revolutionized 3D scene reconstruction by leveraging neural networks for volumetric rendering, achieving unprecedented photorealistic results. Currently, the perceptual quality assessment of NeRF still relies heavily on full-reference (FR) metrics, such as PSNR and SSIM, which require external reference images from predefined camera positions and suffer from significant limitations. In this paper, we propose a new quality metric that directly leverages training views to quantify the perceptual quality of NeRF-generated scenes, eliminating the need for external predefined reference images and camera position metadata. In the proposed approach, we first extracted hierarchical and abstract features from training views using pretrained deep convolutional neural networks (CNNs) and then constructed a reference representation for perceptual quality evaluation through feature space interpolation. We evaluated the effectiveness of the proposed approach with two options: one with pretrained CNNs only (without calibration) and another with calibration applied to learn the importance of hierarchical feature stages. The experimental results demonstrate the effectiveness of the proposed method, which outperforms traditional no-reference (NR) metrics while being comparable to popular FR metrics. We found that deep features trained for high-level classification tasks have a strong potential to quantify perceptual quality across different viewpoints of the same object in NeRF. The code is released at https://github.com/WemoLuo/NVQS