A super-resolution reconstruction method for pixelated multispectral images from division-of-focal-plane sensors

Abstract

Pixel-level division of focal-plane (DoF) sensors enable rapid and simultaneous multimodal imaging but sacrifice images’ resolution, which, for examples, are widely used for polarimetric and/or multispectral imaging applications. As a result, the acquired pixelated images often necessitate resolution enhancements to facilitate more precise information extraction. Existing methods for enhancing the resolutions of DoF images often suffer from some limitations such as the inaccurate reconstruction and the limited applicability. This paper presents a novel super-resolution reconstruction method to enhance the quality of multispectral images captured by pixel-level DoF sensors. The proposed method involves acquiring the reflectance image and extracting the structure image from the raw mosaic image, followed by fusing with the low-resolution multispectral images to generate high-resolution multispectral images. And to make full use of the detailed information of the structural image, the Huber penalty function is selected as the regularization term and introduced into the fusion model. In the experiment, this method obtains an average reconstruction error as small as 0.03 pixel, surpassing seven other existing reconstruction methods; moreover, it also exhibits notable advantages in various evaluation metrics, indicating its success in reconstructing high-resolution multispectral images and preserving both the spatial and the spectral information.