OA-WinSeg: Occlusion-Aware Window Segmentation With Conditional Adversarial Training Guided by Structural Prior Information

Abstract

Window segmentation and vectorization remains a significant challenge, particularly in the absence of clean facade images. To extract complete window segments from building façade images with occlusions, this article proposes an occlusion-aware window segmentation (OA-WinSeg) network with conditional adversarial training guided by prior structural information. This architecture combines the power of image segmentation and generative capabilities to handle occlusions. First, OA-WinSeg automatically detects occlusions and generates a rectangular boundary guidance from a coarse window segmentation, which incorporates structural information about the building layout into the process. Subsequently, the network refines the coarse segmentation and generates window segments in the missing regions by attending to contextual information of the nonoccluded parts of the façade. Finally, our approach generates accurate vector representations, information needed for building modeling systems. Experimental results demonstrate the effectiveness of our model with simulated and occluded real-world datasets. In addition, we evaluate our model on various ablation studies to explore the contribution of the different modules. Finally, we have analyzed the potential applications of the proposed segmentation network and the completed window segments, including building façade inpainting.