CBRFormer: rendering technology-based transformer for refinement segmentation of bridge crack images

High-resolution (HR) imaging devices are crucial for ensuring the safety and efficiency of unmanned aerial vehicles (UAVs) during bridge crack detection tasks. However, due to the limitations of executing sampling discretely in traditional deep learning (DL) architectures and the constraints of GPU computing resources, it is challenging to perform fine-grained segmentation for HR crack images. To effectively address the challenge, the authors drew inspiration from the fine-grained rendering technology in the field of computer graphics (CG) and proposed the Crack Boundary Refinement Transformer (CBRFormer). Through three customized improvements, this architecture fully leverages the advantages of the rendering head in the refined representation of HR crack images. Firstly, a lightweight Transformer-based encoding architecture is designed, enabling the network to accurately capture crack backbone features from complex backgrounds. Subsequently, a boundary-guided branch based on super-resolution reconstruction technology is introduced to assist the network in capturing deep semantic information about crack boundary details. Additionally, two types of refined rendering point sampling methods are tailored for hard example areas during training and inference stages, ensuring that the prediction head used for refined rendering effectively focuses on ambiguous crack boundaries and tiny crack regions. Finally, the effectiveness of each component in the CBRFormer and the network’s practicality are demonstrated through ablation and the field experiment. Compared to the current advanced HR segmentation architectures like CascadePSP and Segfix, the CBRFormer achieved average performance improvements of 2.16% in mean Intersection over Union (IoU), 7.80% in mean Edge Accuracy (mEA), and 2.46% in Dice coefficient, respectively. The utilization of the CBRFormer enables precise segmentation of HR crack images, providing inspectors with more comprehensive and accurate structural crack information, thereby offering technical support for structural safety assessment and maintenance decision-making.