Universal deep demosaicking for sparse color filter arrays

Abstract

Sparse color filter array (CFA) is a potential alternative for the commonly used Bayer CFA, which uses only red (R), green (G), and blue (B) pixels. In sparse CFAs, most pixels are panchromatic (white) ones and only a small percentage of pixels are RGB pixels. Sparse CFAs have the motivation of human visual system and superior low-light photography performance. However, most of the associated demosaicking methods highly depend on synthetic images and are limited to a few specific CFAs. In this paper, we propose a universal demosaicking method for sparse CFAs. Our method has two sequential steps: W-channel recovery and RGB-channel reconstruction. More specifically, it first uses the W channel inpainting network (WCI-Net) to recover the W channel. The first layer of WCI-Net performs the scatter-weighted interpolation, which enables the network to work with various CFAs. Then it employs the differentiable guided filter to reconstruct the RGB channels with the reference of recovered W channel. The differentiable guided filter introduces a binary mask to specify the positions of RGB pixels. So it can handle arbitrary sparse CFAs. Also, it can be trained end-to-end and hence could obtain superior performance but do not overfit the synthetic images. Experiments on clean and noisy images confirm the advantage of the proposed demosaicking method.