Bolt Looseness Quantitative Visual Detection With Cross-Modal Fusion

Abstract

Intelligent bolt looseness detection systems offer significant potential for accurately promptly detecting bolt looseness. Bolt looseness detection in high-speed train undercarriages is challenging due to the low-texture surfaces of structural parts and variations of illumination and viewpoint in typical maintenance scenes. These factors hinder the quantification detection of bolt looseness using traditional 2D visual inspection methods. In this paper, we present a cross-modal fusion-based method for the quantification detection of bolt looseness in high-speed train undercarriages. We propose a cross-modal fusion approach using a cross-modal transformer, which integrates 2D images and 3D point clouds to improve adaptability to varying illumination conditions in maintenance scenes. To address geometric projection distortions caused by varying-view perspective transformations, we use the height difference between the bolt cap and the fastening plane in point clouds as the criterion for bolt loosening. The experimental results indicate that the proposed method outperforms the base-line on our dataset of 5823 annotated RGB-D images from a locomotive depot, achieving an average measurement error of 0.39 mm.