Automated evaluation of degradation in stone heritage structures utilizing deep vision in synthetic and real-time environments

Preserving historical structures stands as a crucial pillar in the realm of sustainable development, where safeguarding our heritage takes precedence. However, the manual monitoring of these architectural marvels as they weather the passage of time remains a tedious and time-consuming endeavor. In this pursuit, computer vision emerges as a transformative tool, poised to alleviate or even eradicate the necessity for extensive human involvement in on-site assessments. The novel contributions of this study to the realm of knowledge encompass two key aspects: firstly, the utilization of synthetic images for categorizing vegetation, and secondly, the adaptation of the model for automated defect identification, culminating in the creation of an algorithm tailored for flaw detection in historical stone buildings. Present research endeavors to harness the prowess of the Mask Region-based Convolutional Neural Network (R-CNN) algorithm, a cutting-edge computer vision methodology, to discern, pinpoint, and delineate deteriorations within historical stone structures, classifying anomalies akin to vegetative growth. The model's training phase involves meticulously annotating anomalies within 501 synthetic images generated through Blender 3D software. Subsequently, the model undergoes rigorous testing, employing 428 photographs sourced from iconic UNESCO World Heritage sites across India, including Udayagiri and Khandagiri Caves, Golconda Fort, and Lingaraj Temple. The dataset expands to encompass a total of 2846 images, partitioned into an 80:10:10 ratio for training, validation, and testing, respectively. The Mask R-CNN-based ResNet101 model yields an impressive accuracy rate of 94 %, coupled with an Intersection over Union (IOU) of 94 %. Meanwhile, the Mask R-CNN-based ResNet50 model achieves a commendable accuracy of 93 %, with an IOU of 91 %. To gauge the efficacy of the proposed model, a comparative analysis is conducted with the state-of-the-art YOLOv8 model. Henceforth, the proposed methodology can be utilized to create an automated system for the visual inspection of stone-built heritage structures, to preserve them for future generations.