Learning hierarchical image feature for efficient image rectification

Abstract

Image stitching methods often use single-homography or multi-homography estimation for alignment, resulting in images with undesirable irregular boundaries. To address this, cropping and image inpainting are the common operations but discard image regions or introduce content that differs from reality. Recently, deep learning-based methods improve the content fidelity of the rectified images, while suffering from distortion, artifacts, and discontinuous deformations between adjacent image regions. In this work, we propose an efficient network based on the transformer (Rectformer) for image rectification. Specifically, we propose the Global and Local Features (GLF) module, which consists of the Hybrid Self-Attention module and Dynamic Convolution module to capture hierarchical image features. We further introduce two auxiliary losses for better image rectification, bidirectional contextual (BC) loss and deformation consistency (DC) loss. The bidirectional contextual loss encourages the model to preserve image local structure information. The loss of deformation consistency improves the network’s geometric recovery and generalization capabilities through a self-supervised learning strategy. Finally, extensive experiments demonstrate that our method outperforms the existing state-of-the-art methods for rotation correction and rectangling.