[Paper] Preserved Color Pixel: high-resolution and high-colorfidelity image acquisition using single image sensor with sub-half-micron pixels

Abstract

The small camera module has been widely implemented in mobile devices such as the smartphone, and its image quality has been consistently improving owing to the progress of the technology and design of small camera module and image sensor device, combined with an advanced image post-processing algorithm supported by low-power and high-performance computing capabilities. Image resolution and color fidelity are part of crucial indices describing the image quality of camera modules employing a single image sensor with a color filter array (CFA), and the sampling frequency of pixel array has been increased by shrinking the pixel pitch while improving the intrinsic pixel performance. The significance of the conventional approach remains unchanged; whereas there will be emerging challenges as the pixel pitch shrinks, given the conditions that both the size of the camera module for the mobile device and the wavelength range of visible light are kept constant. It will be more difficult to confine electromagnetic energy of light by the micro-lens and wave-guide, and its leakage to the adjacent pixels, i.e., a cross-talk, comes to be more evident. The improvement of camera image quality with a single sensor has a possibility to hit a plateau; it is therefore expected to support the continuing improvement trend with an additional, such as computational, approach. With regard to the cross-talk correction, so far, the existing algorithms either assumes a cross-talk Abstract A preserved-color-pixel (PCP) concept is proposed. The PCP color filter array (CFA) is arranged to construct "PCP pixels". A PCP pixel is surrounded by "buffer pixels" having color filters of the same color spectrum as that of the PCP pixel, so that most of color cross-talk from pixels of different colors are absorbed by the buffer pixels. The color cross-talk components of the buffer-pixel signals are computationally canceled by a proposed non-parametric method called "similarity-based blind cross-talk correction (SBC)," where signals of PCP pixels are used as the ground truth to estimate the signals of buffer-pixels without influence of the crosstalk. The demosaicing of each color planes' images sampled with a PCP-CFA arrangement is implemented by the adaptive normalized convolution (ANC) in conjunction with the proposed "post-convolutional-variationminimization (PCVM)" algorithm for its cost function. Both SBC and PCVM-ANC are especially useful for image acquisition with a pixel array in a sub-half-micron generation, where its pixel pitch is approximately, or smaller than, 0.5 μm. The concept is verified with image simulation, and its effectiveness is quantified with the slantededge based spatial frequency response (SFR) modular transfer function (MTF) method by using the parametric color cross-talk analysis based on proposed "scalable-single-parameter (SSP)" color cross-talk model. The image simulation confirms the color reproductivity, together with the effectiveness of image resolution improvement under the influence of the complication of color cross-talk between pixels and lateral chromatic aberration (LCA) of the taking-lens. The benefit is also verified by peak-signal-to-noise-ratio (PSNR) analysis with simulated images based on a real-world picture, indicating that the proposed concept can maintain PSNR when color crosstalk increases.