Bimodal defect segmentation with Geometric Prior-supported Anti-imbalance Learning for pavement defect evaluation and repair

Abstract

Pavement defect segmentation benefits from integrating 2D images with 3D depth data, enabling more accurate evaluation and repair decisions. However, current methods suffer from indiscriminate feature extraction that overlooks severe class imbalance, limited adaptability to defects with diverse shapes and appearances, and a lack of publicly available bimodal datasets with multi-class annotations. To address these challenges, this paper proposes Geometric Prior-supported Anti-imbalance Learning (GPAL), a bimodal segmentation framework based on the Segment Anything Model (SAM). GPAL introduces: (1) Defect-centric Reinforcing (DR) prompting, which leverages geometric priors derived from depth flow to enhance the image encoder’s focus on defect details in a defect-oriented manner; and (2) Depth Prototype-support Uniform Prompting (DPUP), which extends SAM for bimodal, multi-class defect segmentation by learning class-balanced depth prototypes. Additionally, the Bimodal Multi-category Pavement Defect (BMPD) dataset is constructed, containing 5,059 samples across four defect categories. Experimental results demonstrate that the proposed method effectively highlights defects under imbalanced conditions and adapts to diverse defects, achieving an mF1 score of 86.01% on the BMPD dataset. This advancement supports reliable pavement assessment and repair decisions, while laying a foundation for addressing class imbalance in diverse tasks through geometric priors and prompting strategies.