Deep learning-based damage detection and segmentation in the battledore of Darbhanga Fort

Abstract

India's rich and diverse cultural heritage (CH) is mirrored in its architectural treasures, each structure bearing a unique story of history and craftsmanship. This research delves into the application of advanced technologies for the preservation and protection of these invaluable cultural assets. In the present study, You Only Look Once version 8 (YOLOv8), a sophisticated deep learning (DL) architecture, has been meticulously harnessed as a powerful instrument for the precise detection, and detailed segmentation of superficial damages in the form of crack, spalling, and vegetation. As an illustrative case study, we focus our attention on the resplendent Darbhanga Fort, an architectural gem crafted from brick masonry and nestled in the historic heart of Darbhanga, Bihar. The preliminary survey reveals a poignant narrative, one shaped by the absence of consistent upkeep—unveiling a landscape marred by formidable cracks, spalling, and encroaching vegetation. A comprehensive damage dataset of 7440 images (excluding background images) has been meticulously curated from the considered masonry heritage, along with its bounding box labels for precise damage detection and polygon box labels for intricate damage segmentation. This invaluable dataset has been thoughtfully shared with the global research community on the open-source platform of Mendeley's database. The maximum mean Average Precision at Intersection over Union (IoU) threshold of 0.5 (mAP_0.5) of 96.2 % and 92.5 % have been obtained for YOLOv8 detection as well as segmentation models respectively.