A Novel End-to-End Low-Light Image Segmentation Method and Its Application to Robotic Perception

Abstract

In the fields of robotic perception and computer vision, achieving accurate semantic segmentation of low-light or nighttime scenes is challenging. This is mainly due to the limited visibility of objects and reduced texture and color contrasts among them. To address the issue of reduced texture among objects, we propose a dual closed-loop bipartite matching algorithm to establish a total loss function consisting of unsupervised illumination enhancement loss and supervised intersection-over-union loss, thus enabling the joint minimization of both losses via the Hungarian algorithm. We can thus achieve end-to-end training for a semantic segmentation network especially suitable for handling low-light scenes. By utilizing reference daylight images of similar scenes, we propose a weakly supervised method with selective refinement to improve segmentation accuracy in underperforming categories, thus resulting in a model with superior performance. Experimental results show that the proposed network surpasses existing methods on the Cityscapes dataset and notably outperforms state-of-the-art methods on Dark Zurich and Nighttime Driving datasets. We finally deploy the proposed model on quadruped robots to evaluate its performance in outdoor low-light scenes, where it demonstrates superior segmentation capabilities over existing real-time semantic segmentation networks.