MMCFNet: Multi-scale and multi-modal complementary fusion network for light field salient object detection

Abstract

Light field salient object detection (LFSOD) has received growing attention in recent years. Light field cameras record the direction and intensity of light in a scene, and they provide focal stacks and all-focus images with different but complementary characteristics. Previous LFSOD models lack effective feature fusion for multi-scale and multi-modal information, which leads to background interference or incomplete salient objects. In this paper, we propose a new multi-scale and multi-modal complementary fusion network (MMCFNet) for LFSOD. For the focal stacks, we design a slice interweaving enhancement module (SIEM) to emphasize the useful features among different slices and reduce inconsistency. In addition, we propose a new multi-scale and multi-modal fusion strategy, which contains high-level feature fusion module (HFFM), cross attention module (CrossA), and compact pyramid refinement (CPR) module. The HFFM fuses high-level multi-scale and multi-modal semantic information to accurately locate salient objects. The CrossA enhances low-level spatial-channel information and refines salient object edges. Finally, we use the CPR module to aggregate the multi-scale information and decode it into high-quality saliency maps. Extensive experiments on public datasets show that our method outperforms 11 state-of-the-art LFSOD methods.