FormerPose: An efficient multi-scale fusion Transformer network based on RGB-D for 6D pose estimation

Abstract

The 6D pose estimation based on RGB-D plays a crucial role in object localization and is widely used in the field of robotics. However, traditional CNN-based methods often face limitations, particularly in the scene with complex visuals characterized by minimal features or obstructed. To address these limitations, we propose a novel holistic 6D pose estimation method called FormerPose. It leverages an efficient multi-scale fusion Transformer network based on RGB-D to directly regress the object’s pose. FormerPose can efficiently extract the color and geometric features of objects at different scales, and fuse them based on self-attention and dense fusion method, making it suitable for more restricted scenes. The proposed network realizes an enhanced trade-off between computational efficiency and model performance, achieving in superior results on benchmark datasets, including LineMOD, LineMOD-Occlusion, and YCB-Video. In addition, the robustness and practicability of the method are further verified by a series of robot grasping experiments.