CoARF++: Content-Aware Radiance Field Aligning Model Complexity With Scene Intricacy.

Abstract

This paper introduces the concept of Content-Aware Radiance Fields (CoARF), which adaptively aligns the model complexity with the scene intricacy. By examining the intricacies of radiance fields from three perspectives, model complexity is adapted through scalable feature grids, dynamic neural networks, and model quantization. Specifically, we propose a hash collision detection mechanism that removes redundant feature grid by restricting the valid hash collision to reasonable level, making the space complexity scalable. We introduce an uncertainty-aware decoded layer, where simple points are early-exited to prevent them from being processed by deeper network layers, ensuring computational complexity scalable. Furthermore, we propose Learned Bitwidth Quantization (LBQ) and Adversarial Content-Aware Quantization (A-CAQ) paradigms by making the bitwidth of parameters differentiable and trainable, allowing for adjustable quantization schemes. Building on these techniques, the proposed CoARF++ framework enables a scalable pipeline for radiance fields that is tailored to the unique characteristics of scene complexity and quality requirement. Extensive experiments demonstrate a significant and adjustable reduction in model complexity across various NeRF variants, while maintaining the necessary reconstruction and rendering quality, making it advantageous for the practical deployment of radiance field models. Codes are available at https://github.com/WeihangLiu2024/Content_Aware_NeRF.