Foreground-background separation and deblurring super-resolution method

Abstract

The limited depth of field (DOF) inherent in cameras often results in defocused and blurry backgrounds when capturing images in large aperture mode. This not only leads to the loss of crucial background information but also impedes the efficient reconstruction of the background regions. Usually, super-resolution (SR) techniques struggle to produce high-quality results for images captured with large apertures. To enhance the reconstruction quality of defocused regions in large aperture images, a foreground-background separation and deblurring super-resolution (FBSDSR) method was proposed in this paper. Based on the idea of foreground-background separation processing, we first divide the large aperture image into a sharp foreground region (I_f) and a blurry background region (I_b) based on depth information. The background region (I_b) is then deblurred using an end-to-end iterative filter adaptive network (IFAN). This deblurring process refocuses the background, ultimately restoring an image with sharp details throughout. Finally, the enhanced super-resolution generative adversarial networks (Real-ESRGAN) which specializes in images SR of realistic scenes was used to process the sharp all-in-focus image. This method results in high-quality reconstructions of both the foreground and background of large aperture images. The experimental results demonstrated that the proposed method achieved effective reconstruction of the entire large aperture images clearly, overcoming the limitations of existing methods that struggle to reconstruct defocused regions. This significantly enhances the quality and resolution of large aperture images. Specifically, when FBSDSR is integrated with Real-ESRGAN, the PSNR, LPIPS, NIQE, and hyperIQA metrics were improved by approximately 2.2 %, 45.1 %, 34.7 %, and 10.9 % respectively.