A fully pipelined, low-overhead, and energy-efficient CNN-based feature extraction accelerator for mobile visual SLAM

Abstract

Feature extraction is critical for Visual Simultaneous Localization And Mapping (VSLAM). The CNN-based SuperPoint outperforms traditional feature extractors but its high complexity hinders deployment on energy-constrained edge devices like small mobile robots. This paper proposes an energy-efficient SuperPoint hardware accelerator for VSLAM. The key contributions are: (1) developing a lightweight SuperPoint network by reducing filter numbers based on hierarchical feature characteristics, achieving an 88.3 % reduction in model size; (2) implementing a fully pipelined architecture to avoid general-purpose processing and deep learning IP, improving energy efficiency by eliminating off-chip access and sequential computation; (3) introducing a selective descriptor convolution strategy to skip redundant calculations on non-feature points, reducing descriptor computation and hardware overhead; and (4) proposing an optimized Non-Maximum Suppression strategy to remove duplicate comparisons within the sliding windows, further enhancing energy efficiency. FPGA evaluation shows 9.09 × lower hardware overhead and 58.2 mJ/frame energy efficiency, 22.2 % better than state-of-the-art, processing 480 × 640 images at 20 fps under 200 MHz.