DPNet: A dual prototype few-shot semantic segmentation network for crack detection

Abstract

Road crack detection is crucial for maintaining the aesthetics and safety of roads. The varying morphology of cracks often results in insufficient road crack samples, limiting the effectiveness of existing detection methods in few-sample scenarios. Further, when visual samples are insufficient, employing textual information to extract visual information from images is a cutting-edge technology. In this paper, we propose a Dual Prototype Network (DPNet) for few-shot crack detection. Firstly, we introduce an Improved Pixel Weight (IPW) data enhancement to strengthen the foreground and edges of cropped samples, improving learning efficiency in the case of insufficient samples. Next, we design a dual prototype prediction method. Specifically, we employ domain related text input to generate a Language-Image Prototype (LIP) with general domain knowledge through Contrastive Language-Image Pre-training (CLIP). Then, we generate a Support Prototype (SuP) with specialized domain knowledge from crack dataset images. The final prediction is obtained by linearly combining the predictions of the two prototypes. Additionally, we design an Embedding Attention Module (EAM), which leverages the characteristics of the embedding dimension to simultaneously satisfy both spatial and channel attention mechanisms in the transformer structure. Finally, our DPNet achieves superior performance on the FCrack-i and MixCrack few sample datasets, with an average mIoU improvement of 8.52% and 1.44% compared to the baseline. Moreover, we demonstrate the zero-shot capability of DPNet on CFD crack dataset.