FRENet: Feature Relocation Network for Building Segmentation in Remote Sensing Images

Abstract

The challenge of locating target regions owing to the absence of spatial position information in image-level labels is resolved by using class activation maps (CAMs) for building localization. However, due to the intrinsic difference between classification and segmentation tasks, CAM is only able to focus on salient regions and often ignores subsignificant regions. This leads to incomplete internal regions and unclear boundaries of buildings extracted using weakly supervised models. Therefore, we propose an effective weakly supervised building extraction method, FRENet, to solve the above problems. So as to preserve building’s edge information, we employ dual-flow edge compensation module to extract various frequency components of the structure and add an auxiliary branch to provide additional guidance. Moreover, we design a feature sensing module to relocate the building area, which further guides the model to generate more complete CAM. To evaluate the effectiveness of FRENet, we conducted experiments on the Postdam and Vaihingen datasets, comparing it with seven mainstream weakly supervised models. FRENet outperforms AEFT by 2.93% and 4.47%, respectively, on the Postdam dataset, outperforming the competing approaches in terms of F1 score and mIoU. On the Vaihingen dataset, the F1 score and mIoU of FRENet reached 89.91% and 81.75%, respectively, which were 1.19% and 1.94% higher than the second-best ACGC. To further explore the source of this improvement, we visualized and analyzed frequency components of feature maps along with CAMs. The results provide insights into the feature learning advantages of FRENet and empirically validate the effectiveness of its design.