3D detection of pavement cracking utilizing a neural radiation field (NeRF) and semantic segmentation network

Abstract

The automatic detection of pavement cracks is important for maintaining road surfaces and vehicle safety. However, the current two-dimensional image-based automatic pavement detection method has environmental limitations. A novel framework for automatically detecting pavement cracks was proposed that combines computer vision techniques with deep learning methodologies. Using the neural radiance field multi-view stereoscopic imaging algorithm (Point-NeRF), we established a road surface point-cloud model comprising color images, depth images, and color-depth overlapping images. This approach overcame the limitations of the traditional structure from motion-multi-view stereo (SFM-MVS) algorithm through its fast modeling capabilities. To improve the spatial feature extraction structure for semantic segmentation tasks while reducing the network training burden, our proposed deep learning framework was based on PointNet++ with integrated coordinate attention and aggregate attention pooling mechanisms. The results indicate that the average root mean square error of the point pair distance reconstructed by the Point-NeRF algorithm and the point cloud reconstructed by the SVM-MVS algorithm was < 1.4 cm, demonstrating a high level of accuracy. Furthermore, the models had strong similarity, indicating an excellent reconstruction quality and a tenfold improvement in modeling efficiency compared to the SVM-MVS algorithm. Our proposed method achieved an outstanding performance in semantic segmentation tasks for pavement crack detection in terms of overall accuracy (91.8 %), mean intersection over union (80.1 %), precision (89.2 %), recall (87.3 %), and F1 score (88.2 %). Errors in length, width, and depth calculations were within acceptable limits with deviations below 0.3, 0.02, and 0.7 cm, respectively, while error rates were below 20 %.