CTISNeRF: Efficient 4-D Hyperspectral Scene Rendering and Generation With Computed Tomography Imaging Spectrometer

Abstract

The hyperspectral data, renowned for its capacity to provide comprehensive spectral details, are widely applied in a range of low-level and high-level tasks in remote sensing and computer vision. In this article, we introduce an algorithm that, for the first time, leverages snapshot spectral imaging technology to generate 4-D hyperspectral-spatial data, named CTISNeRF. This advancement is made possible through the use of a computed tomography imaging spectrometer (CTIS), a cutting-edge sensor technology capable of capturing high-resolution spectral and spatial information in a single snapshot. In addition, a cutting-edge 360° panoramic hyperspectral dataset has been created and made publicly available. Our approach utilizes data from the CTIS sensor and a zeroth-order feature-sharing mechanism to adeptly learn spectral and spatial characteristics from diverse scenes. This enables the rendering of high-fidelity spectral cubes for novel views, significantly enhancing the quality and detail of hyperspectral imaging. Extensive experimental outcomes demonstrate that CTISNeRF not only markedly reduces the expenses associated with data collection but also achieves superior image quality. It reaches state-of-the-art standards in metrics, such as peak signal to noise ratio (PSNR), structure similarity index measure (SSIM), and learned perceptual image patch similarity (LPIPS). Furthermore, CTISNeRF maintains a more stable generation capability even when the number of training samples is reduced, showcasing its robustness and efficiency. The associated dataset and our algorithm will be publicly accessible at the following repository: https://github.com/YifanSi/CTISNeRF