Snapshot Hyperspectral Imaging With Co-Designed Optics, Color Filter Array, and Unrolled Network

Abstract

We propose a novel snapshot hyperspectral imaging method that incorporates co-designed optics, a color filter array (CFA), and an unrolled post-processing network through end-to-end learning. The camera optics consists of a fixed refractive lens and a diffractive optical element (DOE). The learned DOE and CFA efficiently encode the hyperspectral data cube on the sensor via phase and amplitude modulation at the camera aperture and sensor planes, respectively. Subsequently, the unrolled network reconstructs the hyperspectral images from the sensor signal with high accuracy. We conduct extensive simulations to analyze and validate the performance of the proposed method for several CFA models and in non-ideal imaging conditions. We demonstrate that the Gaussian model is effective for parameterizing the spectral transmission functions of CFA pixels, providing high reconstruction accuracy and being relatively easy to implement. Furthermore, we show that learned CFA patterns are effective when optimally coupled with co-designed diffractive-refractive optics. We evaluate the robustness of our method against sensor noise and potential inaccuracies in the fabrication of the DOE and CFA. Our results show that our method achieves superior reconstruction quality compared to state-of-the-art methods, excelling in both spatial and spectral detail recovery and maintaining robustness against realistic noise levels.