RGB-D Fusion Through Zero-Shot Fuzzy Membership Learning for Salient Object Detection

Abstract

Significant improvement has been achieved lately in color and depth data-based salient object detection (SOD) on images from varied datasets, which is mainly due to RGB-D fusion using modern machine learning techniques. However, little emphasis has been given recently on performing RGB-D fusion for SOD in the absence of ground truth data for training. This article proposes a zero-shot deep RGB-D fusion approach based on the novel concept of fuzzy membership learning, which does not require any data for training. The constituent salient object maps to be fused are represented using parametric fuzzy membership functions and the optimal parameter values are estimated through our zero-shot fuzzy membership learning (Z-FML) network. The optimal parameter values are used in a fuzzy inference system along with the constituent salient object maps to perform the fusion. A measure called the membership similarity measure (MSM) is proposed, and the Z-FML network is trained using it to devise a loss function that maximizes the similarity between the constituent salient object maps and the fused salient object map. The deduction of MSM and its properties are shown theoretically, and the gradients involved in the training of the Z-FML network are derived. Qualitative and quantitative evaluations using several datasets signify the effectiveness of our RGB-D fusion and our fusion-based RGB-D SOD in comparison with the state-of-the-art. We also empirically demonstrate the advantage of employing the novel MSM for training our Z-FML network.