Improved multi-source remote sensing object detection network by geometry consistency constraint and Gaussian distribution alignment

Abstract

Multi-source remote sensing object detection, by combining data from different sensors, can comprehensively improve the accuracy and robustness of object detection. However, it faces challenges such as data inconsistency, domain shift, and scarcity of labeled data. Domain adaptation methods can address these challenges by aligning features between the source and target domains, reducing domain shift, and enhancing the model’s generalization ability, thus solving the discrepancies in multi-source data. However, existing domain adaptation object detection methods insufficiently utilize shallow geometric features that are important for geometric consistency, and traditional methods that use adversarial networks for feature alignment often leading to insufficient alignment capability and training instability. To address the insufficient utilization of geometric information in existing methods and considering that shallow features contain abundant geometric information (e.g., points, lines, and surfaces), this paper proposes a shallow feature alignment method based on geometric consistency (GCFA), using shallow features as alignment cues. This method achieves effective feature alignment through partition calculation and weighted loss processing. Furthermore, to tackle the problems of insufficient alignment capability and training instability in the network, we introduce a feature alignment method based on Gaussian distribution (GDFA). This method directly aligns the feature distributions of the source and target domains by leveraging the mean and standard deviation, thereby enhancing the alignment capability of the network. And we can update the network directly through the loss function, without the need for adversarial networks or gradient reversal layers, thus avoiding potential training instability issues. In addition, we design a pseudo-labels refinement module (PLRM) that combines dynamic threshold select and pseudo-labels class weighting to enhance the constraint ability of the model’s unsupervised branch. In order to verify the effectiveness of the method proposed in this paper, we conducted extensive experiments on datasets such as DOTA, DIOR, WHU, and Levir. On the DOTA and DIOR datasets, the proposed method achieves a 3.09 % improvement in mAP50 compared to the best baseline method. On the WHU dataset, it shows a 2.30 % improvement over the best method, and on the Levir and SSDD datasets, the proposed method outperforms the best method by 2.13 %.