Intrinsic image decomposition based joint image enhancement and instance segmentation network for low-light images

Abstract

Due to low brightness and contrast, low-light conditions present significant challenges for both low-level and high-level vision tasks. For instance segmentation, low-light scenes often result in incomplete objects and inaccurate edges. Existing methods typically treat low-light enhancement as a preprocessing step, adopting an “enhance-then-segment” pipeline, which reduces segmentation accuracy and neglects the information generated during segmentation that is useful for low-light image enhancement (LLIE). To address these issues, we propose a novel strategy that couples LLIE with instance segmentation in a cross-complementary manner, allowing them to mutually improve each other. Specifically, we first replace traditional “enhance-then-segment” approach with a “decompose-then-segment” method by using the reflectance map generated during the enhancement process as input for instance segmentation. The details in the reflectance map can be preserved by improving decomposition loss functions, thus increasing the segmentation accuracy. Then we incorporate instance-level semantic information from the segmentation process with the proposed semantic feature fuse block (SFFB). It integrates semantic information into the feature representation space, guiding the enhancement process to perform differential enhancement on regions based on their semantic content. In addition, we propose an instance-guided color histogram (ICH) loss function to maintain color consistency between the enhanced image and the ground truth across instances. Extensive experiments on LIS dataset demonstrate the effectiveness and generality of our method.