Using the multimodal image transformation method for crack detection in the presence of shadow

Pavement cracks are one of the manifestations of early road defects, which affect the safety and performance of roads. However, shadow, illumination, and other environmental factors will affect the detection performance of the algorithm, resulting in the reduction of the detection effect. This paper presents a multimodal image transformation technology to reduce the interference of shadows on crack detection. In this method, multidimensional features of infrared and visual images are obtained and registered by the multiscale registration method. Next, the Cycle-Gan model is used to learn the feature mapping relationship between infrared and visual images, and image transformation is carried out. Finally, the pre-trained U-net model is used to detect the processed image. The proposed method can effectively reduce the image's shadow area without destroying the pavement's crack structure and improve the detection performance compared with threshold-based image processing and deep learning-based DSC algorithm. In addition, its MIOU, MPA, Recall, and F1 score reach the highest of 0.7869, 0.93295, 0.7092, and 0.7870, respectively, which can provide new ideas for health detection of pavement cracks under shadow interference.