ShARP-WasteSeg: A shape-aware approach to real-time segmentation of recyclables from cluttered construction and demolition waste

Abstract

Instance segmentation is the fundamental computer vision task that facilitates robotic sorting by localizing object instances. This task becomes particularly challenging when dealing with Construction and Demolition Waste (CDW), as CDW objects often exhibit complex, non-uniform shapes and are frequently overlapped or occluded due to cluttering. Current waste segmentation benchmarks relying on fully connected networks for pixel-wise classification overlook crucial shape and boundary information. It is imperative to use shape information to guide mask prediction in order to improve waste segmentation accuracy. In response, this paper introduces ShARP-WasteSeg; a Shape-Aware Real-Time Precise Waste Segmentation framework. This conceptually straightforward approach mutually learns objects masks and boundaries within a single network, resulting in sharper mask predictions for complex recyclables despite cluttering. ShARP-WasteSeg enhances the segmentation process by extracting boundary features from depth maps, which are rich in shape and location information. These features complement RGB boundary features, guiding the final mask predictions through feature fusion. Moreover, it leverages the ground-breaking capabilities of cross-stage partial networks to optimize the feature extraction process, permitting real-time applicability of the multi-modal approach. Tested on a challenging CDW dataset representing real conditions, ShARP-WasteSeg improved Mask Average Precision (AP) by 7.91%, and the boundary-sensitive Boundary Average Precision by a significant 11.44%, demonstrating the effectiveness of the proposed shape-aware approach in increasing boundary quality of predicted masks for cluttered CDW recyclables.