HFNet: High-precision robotic grasp detection in unstructured environments using hierarchical RGB-D feature fusion and fine-grained pose alignment

The ability of robots to accurately grasp objects in unstructured environments is critical for advancing robotic automation. Current grasp detection methods overlook the distinct semantic and spatial attributes across RGB-D feature layers in such scenarios. Additionally, using Intersection over Union (IoU) loss for pose alignment fails to adequately address variations in grasp angles and dimensions. To resolve these limitations, we propose HFNet, a novel grasp detection network integrating hierarchical RGB-D feature fusion and fine-grained pose alignment strategies to optimize grasp configurations, thereby enhancing robotic applicability in real-world unstructured environments. Specifically, we design: (1) a Relation-aware Cross-modal Feature Fusion (RCFF) module to capture mid-level features (object shapes, edges, textures); (2) a High-level Cross-modal Feature Fusion (HCFF) module to strengthen global shape and semantic relationships. A new fine-grained pose alignment loss further improves measurement precision while reducing low-quality grasps. HFNet achieves state-of-the-art accuracy on benchmark datasets (Cornell, Jacquard) and challenging datasets (CBRGD, OCID_grasp, WISDOM), with a 96.67% success rate in physical robotic experiments. The source code and experiment video are available at: https://github.com/meiguiz/HFNet.