Video Snapshot Compressive Imaging via Optical Flow

Abstract

Video Snapshot compressive imaging (SCI) reconstruction recovers video frames from a compressed 2D measurement. However, frames at each time cannot be observed since the limitation of hardware. To make SCI suitable for more applications, we propose an optical flow-based deep unfolding network for video SCI reconstruction. To extract the optical flow, the feature maps during the iterative process are transformed by the convolution layer into the estimated optical flow. We designed a motion regularizer, which uses voxels of iterative frames and optical flow to update the reconstructed frames. The proposed motion regularizer efficiently captures the temporal correlation between the previous and next frames, which contributes to reconstructing the observed and unobserved frames from input measurement in a SCI reconstruction process. Experiments show that our method achieves state-of-the-art results on PSNR and SSIM.