Segmentation and Quantification of Surface Defects in 3D Reconstructions for Damage Assessment and Inspection

Abstract

Detecting surface defects is crucial for maintaining the integrity of critical infrastructure. Traditional RGB image-based methods are limited by their reliance on 2D information, which impairs accurate damage assessment. This paper introduces a novel approach that enhances defect detection and quantification, utilizing dense 3D reconstructions generated through techniques like photogrammetry or profilometry. We develop an improved robust spline fitting algorithm to estimate the undamaged surfaces from the 3D reconstructions. The residual distances between the observed and fitted surfaces are subsequently used to segment and quantify defects. By leveraging 3D data, our method resolves visual ambiguities and enables damage quantification using physically meaningful metrics. For 3D models based on optical sensing, our method complements RGB image-based defect detectors and classifiers, facilitating the fusion of visual and 3D information for a more comprehensive defect analysis. Validated on both synthetic and real-world datasets, our method demonstrates strong performance and practical feasibility. Note to Practitioners—Our research is driven by the growing potential of drone-based inspections using high-resolution imaging platforms, which offer significant advantages for monitoring remote or hard-to-reach infrastructure. With high-quality images, we can use photogrammetry to reconstruct accurate 3D models directly from inspection images, without requiring additional sensors or manual intervention. We leverage this 3D data to improve the robustness of automated defect detection and enable the precise quantification of defect sizes and material loss. Tracking these metrics over multiple inspections can provide valuable information for preventive and predictive maintenance, moving us closer to efficient, comprehensive structural health monitoring. While our method is highly effective across various surfaces, it is less suited to detecting widespread shallow damage, which could benefit from incorporating additional geometric constraints.