RDD: Improving Structure from Motion with a Robust Feature Detector and Descriptor

Abstract

Abstract Robust feature detection and description are core components of 3D vision pipelines, yet they remain challenging under large viewpoint, scale, and appearance changes. Recent methods have improved local feature learning, but many still rely on convolutional operators whose local receptive fields limit their ability to capture long-range geometric cues. We present Robust Deformable Detector (RDD), a robust keypoint detector and descriptor that uses deformable transformers to capture global context and geometric invariance. Deformable attention allows the descriptor branch to focus on a sparse set of informative locations, reducing the effective search space while preserving the ability to model long-range relationships. We also collect an Air-to-Ground dataset for training and evaluation, complementing the standard MegaDepth dataset. RDD outperforms state-of-the-art keypoint detection and description methods in sparse matching and also supports semi-dense matching. To support comprehensive evaluation, we introduce two challenging benchmarks: MegaDepth-View, which emphasizes large viewpoint and scale variations, and Air-to-Ground, which targets cross-altitude matching for 3D reconstruction. Project page: https://xtcpete.github.io/rdd/ .