A central difference attention multi‐modal segmentation network integrating continual learning and graph convolution diffusion algorithms for complex road crack segmentation

Abstract

The presence of vehicles and traffic signs in complex scenarios poses significant challenges for road crack detection. To address these challenges, this paper integrates image and text information and proposes a new cross‐modal road crack detection model, CDGC‐TNet. The model uses a classic encoder–decoder structure for image feature extraction and BERT‐VisTrans text feature extractor for text feature extraction. First, the centered difference attention module is employed to deal with complex background interference. Second, the graph diffusion depth propagation algorithm is used to address the issue of fine cracks in segmentation problems. Finally, we employ a continuous learning mechanism based on flexible memory fusion to address catastrophic forgetting in the model. Through experimental validation on multiple public datasets, CDGC‐TNet outperforms 10 existing advanced crack segmentation networks in all metrics, demonstrating excellent performance and good generalization ability. Tests in real‐world road scenarios further prove the effectiveness of the proposed method, which can provide an efficient and reliable auxiliary tool for road safety detection.