Image inpainting via Smooth Tucker decomposition and Low-rank Hankel constraint

Abstract

Image inpainting, aiming at exactly recovering missing pixels from partially observed entries, is typically an ill-posed problem. As a powerful constraint, low-rank priors have been widely applied in image inpainting to transform such problems into well-posed ones. However, the low-rank assumption of original visual data is only in an approximate mode, which in turn results in suboptimal recovery of fine-grained details, particularly when the missing rate is extremely high. Moreover, a single prior cannot faithfully capture the complex texture structure of an image. In this paper, we propose a joint usage of Smooth Tucker decomposition and Low-rank Hankel constraint (STLH) for image inpainting, which enables simultaneous capturing of the global low-rankness and local piecewise smoothness. Specifically, based on the Hankelization operation, the original image is mapped to a high-order structure for capturing more spatial and spectral information. By employing Tucker decomposition for optimizing the Hankel tensor and simultaneously applying Discrete Total Variation (DTV) to the Tucker factors, sharper edges are generated and better isotropic properties are enhanced. Moreover, to approximate the essential rank of the Tucker decomposition and avoid facing the uncertainty problem of the upper-rank limit, a reverse strategy is adopted to approximate the true rank of the Tucker decomposition. Finally, the overall image inpainting model is optimized by the well-known alternate least squares (ALS) algorithm. Extensive experiments show that the proposed method achieves state-of-the-art performance both quantitatively and qualitatively. Particularly, in the extreme case with 99% pixels missed, the results from STLH are averagely ahead of others at least 0.9dB in terms of PSNR values.