Simultaneous detection and restoration of building rooftop tree occlusion with a self-supervised diffusion process

Abstract

Building rooftops in high resolution remote sensing images often suffer from various occlusion that destroy the original features. However, there is a lack of a comprehensive method for the simultaneous detection and restoration of such occlusions. This paper focuses on tree occlusion and proposes a diffusion-based model, named Rooftop Tree Detection and Restoration (RTDR). The method defines tree occlusion restoration as a T-step denoising process. We innovatively perform occlusion location extraction and original pixel prediction simultaneously. Based on the prediction results of the tree occlusion decomposition model, the gradient of pixel changes within the occluded areas is obtained. This gradient is incorporated into the backward denoising process of the conditional diffusion model to guide the self-supervised pre-trained diffusion model in restoring the complete building rooftop from the occluded image. Meanwhile, this paper proposes a tree occlusion simulation process based on the spatial combination of randomness between rooftops and trees for generating realistic rooftop occlusion data. The experimental results demonstrate that RTDR achieves satisfactory restoration performance on both simulated and real rooftop tree occlusion datasets. On the simulated tree occlusion dataset, the accuracy evaluation metrics PSNR/SSIM/NIQE are 21.736/0.8177/9.1711, respectively; on the real tree occlusion dataset, the quantitative evaluation metrics Precision/Recall/IoU/F1-Score are improved from 0.8568/0.5789/0.5565/0.6656 to 0.8261/0.7863/0.6818/0.7871. In addition, module and sample ablation experiments validate the effectiveness of the spectral rooftop dataset BUCEA4.0 and the robustness of RTDR. Codes and datasets open source at https://github.com/GHLJH/RTDR and https://www.dxkjs.com/tw/Public/about/html/rs_yangben.html.