RTDSeg: Hard example sampling driven Real-Time Concrete Structural Damage Segmentation network

Concrete material will gradually lose its original structural strength over time and suffer from a variety of structural damages, such as cracks, potholes, etc. Diverse damage patterns and complex geometries of material make accurate multi-class material structural damage segmentation more difficult than the segmentation of a single type of damage. Integrating detection methods with other systems and applying them to engineering practice imposes demands on the efficiency of model inference. In response to these challenges, Real-Time concrete structural Damage Segmentation network (RTDSeg) was proposed. In this network, efficient feature extraction backbone was introduced to improve the perceptual capabilities of the model. In order to alleviate the problem of feature redundancy when fusing features from different scales, semantic enhancement module was designed to filter the encoding features. Furthermore, auxiliary prediction head and hard example sampling training method were introduced to optimize the training effectiveness of the model, which improved the model’s prediction accuracy without extra inference cost. A series of experiments demonstrated the superiority of RTDSeg and the effectiveness of several improvements. In the compared state-of-the-art networks, RTDSeg achieved 8.98% mIoU and 13.89% FPS lead on a bridge damage dataset, and 3.88% mIoU and 92.03% FPS lead on a reinforced concrete damage dataset compared to the ones with the highest accuracy.