The extreme performance of video snapshot compressive imaging under system noise constraints

Abstract

Video Snapshot Compressive Imaging (SCI) aims to capture high-speed scenes with low-speed cameras in a low-cost and low-bandwidth manner. Specifically, a high-speed scene is encoded by different modulation masks and then summed up to generate a snapshot compressed measurement which is finally captured by a traditional low-speed camera. Following this, reconstruction algorithms are correspondingly designed to retrieve the compressed dynamic scene. Existing video SCI reconstruction algorithms have achieved superior performance in the simulated testing data and real testing data with less noise. However, in the applications of real imaging systems, the existence of the intrinsic noise within detector results in the mismatch between the simulated and real imaging systems. Therefore, intrinsic noise and light intensity become the major challenges for video SCI reconstruction in the real cases. Bearing the above in mind, in this paper, we propose to integrate the intrinsic noise of the real imaging system into the whole reconstruction pipeline. More importantly, based on the noise-integrated framework, we evaluate the reconstruction performance under different light conditions and compression ratios. Experimental results show that with different signal-to-noise ratios, there exists an extreme performance bound that is lower than that of the noise-free condition. To verify the effectiveness of our proposed method, we build a real video SCI system and carefully calibrate its intrinsic noise. Following this, existing state-of-the-art reconstruction method EfficientSCI is used to present the reconstruction results. Introducing calibrated intrinsic noise significantly improves reconstruction quality under noisy and insufficient-light conditions, bringing performance close to that of a noise-free scenario. The proposed method has further been verified by the experimental results.