PFRNet: Progressive multi-scale feature fusion and refinement for RGB-D salient object detection

Abstract

RGB-D salient object detection, through the integration of multi-modal feature information, is adept at generating visually compelling saliency maps. Despite the advancement of various RGB-D salient object detection models, significant challenges such as detection omissions, inaccurate object localization, and false detections persist, particularly in multi-object environments or cluttered backgrounds. To address these issues, we introduce a Progressive Multi-Scale Feature Fusion and Refinement Network (PFRNet) based on an encoder–decoder architecture. During the feature encoding phase, we utilize a dual-stream Pyramid Vision Transformer as the encoder to extract RGB and depth features. Given that low-level features contain detailed spatial information while high-level features encapsulate semantic information, we adopt the Spatial Detail Aggregation Module (SDAM) and the Semantic Feature Enhancement Module (SFEM) to facilitate the cross-modal fusion of these features. In the feature decoding stage, we design a progressive decoder anchored by the Feature Focusing and Refinement Module (FFRM). This decoder incrementally concentrates and refines discriminative information from fused features at multiple scales, simultaneously eliminating redundant content to achieve precise prediction of salient objects. The experimental results show that PFRNet outperforms 14 existing RGB-D salient object detection models across six public datasets, while demonstrating the method’s strong generalization capabilities in RGB-T salient object detection tasks.