Q-DRFormer: Query-Position Mapping Flow Prediction with Progressive Refinement for Document Rectification

Abstract

Document image rectification aims to restore images degraded by geometric distortions. Existing U-shaped encoder-decoder methods rely on hierarchical upsampling operations to recover dense flow fields, which provide limited control over the displacement at individual grid positions, leading to spatial inconsistency and poor detail restoration. To address this issue, we propose Q‑DRFormer, a novel framework that formulates rectification as a set prediction task. Specifically, we design a set of queries, where each query corresponds to a position in the flow field grid and is responsible for regressing a 2D anchor point that indicates the sampling position in the distorted input. This one-to-one design establishes a deterministic query-position mapping, enabling fine-grained control and improving the spatial consistency of the predicted flow field. Exploiting this deterministic correspondence, we further propose a progressive flow refinement strategy that progressively refines anchor points through layer-wise offset prediction to enhance fine-grained restoration. Moreover, considering the lack of downstream-task-oriented evaluation in existing studies, we apply the rectification method to scene table detection and contribute a real-world test set along with corresponding evaluation metrics to fill this gap. Extensive experiments demonstrate that Q-DRFormer outperforms existing methods. The code is available at https://github.com/vvvvvair/Q-DRFormer.