Sam-based instance segmentation models for the automation of structural damage detection

Abstract

In infrastructure asset management, monitoring structural condition is vital for safety and cost-efficiency. Traditional visual inspections are subjective, inconsistent, and time-consuming. Advanced automating visual inspections using digital technologies and artificial intelligence can effectively address these issues. Previous studies mainly focused on concrete structures and pavements, neglecting masonry defects and lacking publicly available datasets. In this paper, we address these gaps by introducing the “MCrack1300” dataset, annotated for bricks, broken bricks, and cracks, targeting instance segmentation. We propose two novel, automatically executable methods based on the latest visual large-scale model, the prompt-based Segment Anything Model (SAM). We fine-tune SAM’s encoder using Low-Rank Adaptation (LoRA). The first method connects SAM’s encoder to other decoders directly, while the second uses a learnable self-generating prompter. We modify the feature extractor for seamless integration of these methods with SAM’s encoder. Both methods outperform the state-of-the-art models, improving benchmark results approximately 3 % across all classes and around 6 % specifically for cracks. Building on successful detection, we then propose a monocular-based method to automatically convert images into orthographic projection maps via Hough Line Transform. By incorporating known real sizes of brick units and employing Euclidean Distance Transform, we accurately estimate crack dimensions, with the error less than 10 %. Overall, we offer reliable automated solutions for masonry crack detection and size estimation, which effectively enhances the management and maintenance efficiency of masonry structural asset.