DBF-Net: A Deep Bidirectional Fusion Network for 6D Object Pose Estimation with Sparse Linear Transformer

Abstract

6D object pose estimation, a critical component in computer vision and robotics domains, involves determining the 3D location and orientation of an object relative to a canonical reference frame. Recently, the widespread proliferation of RGB-D sensors has precipitated a marked increase in interest towards 6D pose estimation leveraging RGB-D data. A deep bidirectional fusion network is developed, DBF-Net, achieving efficient yet accurate 6D object pose estimation. Specifically, a sparse linear Transformer (SLT) with linear computation complexity is introduced to effectively leverage cross-modal semantic resemblance during the feature extraction stage, such that it fully models semantic associations between various modalities and efficiently aggregates the globally enhanced features of each modality. Once acquiring two feature representations from two modalities, a feature balancer (FB) based on SLT is proposed to adaptively reconcile the importance of these feature representations. Leveraging the global receptive field of SLT, FB effectively eliminates the ambiguity induced by visual similarity in appearance representation or depth missing of reflective surfaces in geometry representations, thereby enhancing the generalization ability and robustness of the network. Experimental results demonstrate that DBF-Net surpasses current state-of-the-art works by nontrivial margins across multiple benchmarks. The code is available at https://github.com/Mrfanxuan/dbf_net.