Joint intensity-spectral polarization hierarchical fusion guided efficient transparent object detection

Abstract

The field of object detection has emerged as a critical and valuable research frontier. Nevertheless, the detection of transparent objects remains an unresolved and challenging problem, primarily due to their limited texture and color information. Towards being able to address this situation, we propose a novel intensity-spectral polarization fusion framework, termed as FuseISP, specifically designed for transparent object discrimination. FuseISP starts by utilizing hierarchical feature extractor for each feature source, i.e., trichromatic intensities or trichromatic linear polarization cues, to produce abundant high- and low-frequency features. Subsequently, we implement an intensity-spectral polarization mixed modulator to enhance interactions between intensity and spectral polarization information. Additionally, FuseISP introduces a new hierarchical feature fusion module to establish connections among different levels for modelling the shared information. Lastly, a multi-level decoder module based on the integration of 2D convolutional neural networks (CNNs) and 3D CNNs, which can simultaneously capture inter- and intra-polarization relationships, is designed to construct the transparent object detector in a deeply supervised manner. Experimental results show our proposed method outperforms other advanced approaches in the real-world scenes.